Showing posts with label AHRQ. Show all posts
Showing posts with label AHRQ. Show all posts

Monday, August 27, 2012

Health IT Amateurs, Logical Fallacy and Hospital Leadership: An Inhibitor to Good Health IT

An excellent three-part article on local providers' efforts to "join the electronic medical record/clinical IT movement", including a middle section "One doctor has reservations about EMR's design, usability", was published yesterday in the Dubuque (IA) Telegraph Herald.

I was cited as that doctor (subscription required).

The article began:

Dr. Scot Silverstein travels the country attending conferences [and other countries as well - ed.], speaking on panels and voicing concerns about health care's headlong rush into a reliance on electronic medical record systems.

"Headlong rush" is an accurate description of my beliefs, as in my "cart before the horse" posts here.

The newspaper reporter continues:

"Older and younger physicians alike are increasingly concerned about the poor design and poor usability of clinical IT," Silverstein said.  

Not only that, but so is ONC, the Institute of Medicine and the National Institute for Standards and Technology, among others, who I indicated were the source of my quotes.

The Drexel University College of Information Science and Technology faculty member calls "EMR" an anachronistic term from a time when the systems were merely storage tools for records.  "What is meant in 2012 is not just an innocuous 'filing cabinet,' but an enterprise clinical resource management and workflow control system - not just storing records, but regulating and governing all clinical behavior and action," Silverstein said.

That is, indeed, my own observation.

Silverstein contends such systems are inappropriate for some health-care environments, such as emergency rooms and intensive-care units.  "They slow down and distract clinicians due to their generally poor user interfaces, in the worst possible setting, and disrupt clinician cognition," he said.

Again, not only me.  As I had written to the reporter:

... These devices are not appropriate for high-risk, high-intensity environments such as ED's, ICU's etc.  They slow down and distract clinicians due to their generally poor user interfaces, in the worst possible setting, and disrupt clinician cognition.  But don't take my word for it.  See the 2012 report from the Institute of Medicine on healthcare IT safety (I wrote about it at http://hcrenewal.blogspot.com/2011/11/iom-report-on-health-it-safety-nix-fda.html), the National Institute of Standards and Technology report on clinical IT usability (http://hcrenewal.blogspot.com/2011/10/nist-on-ehr-mission-hostile-user.html) and the literature I collated at http://www.ischool.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=readinglist.

In fact, document image management systems and human data abstractors are a good tradeoff to meet the needs of the most time-pressed clinicians whose time is a hospital's most valuable asset, and to make patient charts available when and where needed.  This is as opposed to "digital data field and form-based" (i.e., conventional) EMR's that force the clinicians to waste their time in clerical functions and distract them from pressing clinical matters and informational accuracy. 

He then reports:

Silverstein defines such cognition as the decision making and problem solving necessary to provide quality health care.

Precisely.

... Silverstein thinks current clinical IT programs focus too much on raw data and not enough on supporting a physician's decision-making abilities. "As a result, valuable time and energy is spent managing data as opposed to understanding the patient," he said. "Ideally, IT systems would place raw data into context with current medical knowledge to provide clinicians with computer models that depict the health status of the patient, including information on how different organ systems are interacting, epidemiological insight into the local prevalence of disease and potential patient-specific treatment regimens."
 
Actually, it's not just that I think and said these things.  The quote came verbatim, as I had indicated, from the prestigious National Research Council of the United States and their 2009 study on health IT, led by healthcare IT pioneers Drs. Octo Barnett and William Stead.

Any time logically consistent, ethics-based, common sense observations and opinions are expressed about health IT, however, one can always rely on a pundit or hospital executive for a misdirecting, illogical, and/or impertinent comment.

The expected came from Kay Takes, vice president of patient care services at Mercy Medical Center-Dubuque:

 ... Kay Takes, vice president of patient care services at Mercy Medical Center-Dubuque, said the hospital finds electronic medical records a help in the critical environment of the ICU.

"Specifically in the ICU, in the last 13 months we've gotten enhancements that allow us to download values from the medical equipment - it's automatically pulled into the medical record," she said "It's been fantastic. The availability of the information is enormously valuable. It's been a lot more of a benefit than a hindrance."

I speak from experience, in having been involved in developing those exact same capabilities in the mid 1990's (or, most accurately, protecting patients from the dangers created by the IT department in the project), that they are a convenience to those who formerly had to collect the data manually and write it on the ICU flowsheet.

The capabilities are also a mild convenience to clinicians who view the data, although the surface area of a paper flowsheet is a great advantage in seeing more data in one's field of view at one time than the usual small computer screen (to illustrate see my Feb. 2012 post EHR Workstation Designed by Amateurs at this link and my Jan. 2012 post An 'Anecdotal Complaint' About An ICU EHR at this link).

From the latter post:

... And we do still talk to each other – but even that doesn’t always “work out”, because we’ve lost our operational minds (collectively) – the shared-by-all compact, visually all data in one place, and temporally arranged – i.e., the shared nurse/doc/resp flowsheet [traditionally in an ICU, a long tabular scroll of paper for "at a glance" patient status overview - ed.] – where everybody was looking at the same page, which we no longer are – as the team is slowly discovering.

And which required no logon for sign-over bedside rounding (~40 minutes for 20-30 babies was the allotted time). The flowsheet needed only a 10-15 second glance to spot developing problems; “the computer” is effectively inaccessible in the time allotted for the twice daily sign-n-out “rounds”.

Ultimately, though, Ms. Takes, if quoted accurately, commits the logical fallacy of ignoratio elenchi ("ignorance of refutation", missing the point) – an argument that may in itself be valid, but does not address the issue in question.

For this convenience does not at all justify the downsides of EHRs, especially in an ICU:  increased time for task completion, increased risk of errors of commission or omission, and the other risks as outlined in sources such as FDA's 2010 Internal Memo on H-IT Risks, and recently by AHRQ in their IT Hazards Manager project (Appendix B).

Let's review those risks and failure modes from the AHRQ report, all observed empirically in the real world.  From the report:

A health IT hazard is a characteristic of any health IT application or its interactions with any other health care system (e.g., the people, equipment and work spaces of an ICU) that increases the risk that care processes will be compromised and patients harmed.

The potential outcomes of these hazards include medical privacy breach/identity theft, medical misadventures such as errors of commission or omission resulting in "close calls" (errors barely averted), or patient injury, or death, stress on clinicians reducing their performance, and documentation errors or data corruption increasing the risk of errors in the short, medium or long term.  In short, nothing you'd likely desire to occur while you or a family member was a patient:

Usability
• Information hard to find
• Difficult data entry
• Excessive demands on human memory
• Sub-optimal support of teamwork (situation awareness)
• Confusing information display
• Inadequate feedback to the user
• Mismatch between real workflows and HIT
• Mismatch between user mental models/expectations and HIT

Data Quality
• IT contributed to entry of data in the wrong patient’s record
• Organizational policy contributed to entry of data in the wrong patient’s record
• Patient information/results routed to the wrong recipient
• Discrepancy between database and displayed, printed or exported data
• Faulty reference information
• Unpredictable elements of the patient’s record available only on paper/scanned documents
• Lost data
• Inaccurate natural language processing
• Virus or other malware

Decision Support
• Excessive non-specific recommendations/alerts
• Faulty recommendation
• Missing recommendation or safeguard
• Inadequate clinical content
• Inappropriate level of automation

Vendor Factors

• Sub-optimal interfaces between applications and devices
• Faulty vendor configuration recommendation
• Unusable software implementation tools
• Non-configurable software
• Inadequate vendor Testing
• Inadequate vendor software change control
• Inadequate control of user access
• Faulty software design (specification)

Local Implementation

• Faulty local configuration or programming
• Inadequate local testing
• Inadequate project management
• Inadequate software change control
• Inadequate control of user access
• Suboptimal interface management

Other factors
• Inadequate training
• Excessive workload (including cognitive)
• Inadequate organizational change management
• Inadequate management of system downtime or slowdown
• Unclear policies
• Compromised communication among clinicians (i.e., during handoffs)
• Interactions with other (non-HIT) care systems
• Physical environment (e.g., hardware location, lighting, engineering)
• Inadequately secured data
• Hardware Failure
• Use error in the absence of other factors

The convenience of automated data collection through an EMR system comes at, one might say, a slight cost that may not be realized by health IT amateurs.** 

Unfortunately, their lack of knowledge of these issues reduces the caution and "pushback" required for good health IT to become the norm, and permits bad HIT to be sold.


-- SS

** Amateurs in the sense that I am a radio amateur, not a professional, formally trained telecommunications engineer, and would never take a major role in an enterprise telecommunications project, especially a mission-critical one.

8/27/12  Addendum:

A typical, anonymous, irrational response of the type we've seen on this blog before (as for example here and here) has been posted as a comment to the newspaper story:

"nemesis" posted at 1:19 pm on Mon, Aug 27, 2012. 

He travels the country complaining about existing systems? Perhaps the good prof could spend a little time designing the ideal interface/system and see if he can get anyone interested. Surely his design would win them all over.

The problem with this type of comment, of course, is that it's logically fallacious (a form of ad hominem and an odd "appeal to perfection") and irrelevant.  It is no way responds to or refutes the issues I raise in the article.   

It likely comes from a health IT pundit of some sort, but hopefully not someone in a position of authority.  In medicine, the rule of thumb is simple:  "Critical thinking always, or your patient's dead."

-- SS

Tuesday, June 5, 2012

Cart Before the Horse, Part 3: AHRQ's "Health IT Hazard Manager"

In a July 2010 post "Meaningful Use Final Rule: Have the Administration and ONC Put the Cart Before the Horse on Health IT?" and an Oct . 2010 post "Cart before the horse, again: IOM to study HIT patient safety for ONC; should HITECH be repealed?" I wrote about the postmodern "ready, fire, aim" approach to health IT:

In the first post, I wrote:

... These "usability" problems require long term solutions. There are no quick fix, plug and play solutions. Years of research are needed, and years of system migrations as well for existing installations.

Yet we now have an HHS Final Rule on "meaningful use" regarding experimental, unregulated medical devices the industry itself admits have major usability problems, along with a growing body of literature on the risks entailed.
For crying out loud, talk about putting the cart before the horse...

Something's very wrong here...

However, this situation is anything but humorous.

How more "cart before the horse" can government get?

In the second post, I wrote:

... So, in the midst of a National Program for Health IT in the United States (NPfIT in the U.S.), with tens of billions of dollars earmarked for health IT already (money we don't really have, but it can be printed quickly, or borrowed from China) the IOM is going to study health IT safety, prevention of health IT-related errors, etc. ... only now?

Here we go yet again.

The problem with the AHRQ (Agency for Healthcare Research and Quality, a division of HHS) announcement below of a webinar about a new tool for identifying, categorizing, and resolving health IT hazards, as I have written before, is putting the "cart before the horse" and throwing medical ethics to the wind.

If we've just developed a tool "for identifying, categorizing, and resolving health IT hazards", the magnitude of which others such as IOM admit are unknown to our detriment (e.g., Health IT and Patient Safety: Building Safer Systems for Better Care, pg. S-2), then health IT is, it follows, an experimental technology.

If it is an experimental technology, AHRQ and others in HHS should probably be raising the issue of a slow down or moratorium on widespread rollout under HITECH until risk management and remediation is better understood.  At the very least they should be calling for patient informed consent that a device that will largely regulate their care is experimental, that a competency "gap" exists among healthcare practitioners within the "health IT environment" (meaning patients are at risk), and that patients should be offered the opportunity for informed consent with opt-out provisions.  The principals should not just be announcing a webinar:

Sent: Tuesday, June 05, 2012 12:23 PM
To: OHITQUSERS@LIST.NIH.GOV
Subject: Register Now! AHRQ Health IT Webinar "Purpose and Demonstration of the Health IT Hazard Manager and Next Steps" June 11, 2:30 PM ET

Agency for Healthcare Research and Quality

Purpose and Demonstration of the Health IT Hazard Manager and Next Steps

June 11, 2012 — 2:30-4 p.m., EST

The Agency for Healthcare Research and Quality (AHRQ) has identified a gap in a health care/public health practitioner’s competency within the health IT environment. This webinar is designed to increase practitioners’ competencies in several areas: improving health care decision making; supporting patient-centered care; and enhancing the quality and safety of medication management by improving the ability to identify, categorize, and resolve health IT hazards.

The Webinar will explore the Health IT Hazard Manager—a tool for identifying, categorizing, and resolving health IT hazards. When implemented, the tool allows health care organizations and software vendors alike to learn about potential hazards and work to resolve them, including the use of data to communicate potential and actual adverse effects. The session will discuss how the Health IT Hazard Manager was tested and refined as well as strategies and implications for deploying it. The target audience includes AHRQ grantees/researchers; health care providers, including physicians and nurses; consumers/patients; and health care policymakers.

... Webinar learning objectives include:

1. Describe the rationale for developing the Health IT Hazard Manager and how it evolved through alpha and beta testing.
2. Explain the process for identifying and categorizing health IT-related hazards.
3. Demonstrate how the Health IT Hazard Manager would be used [i.e., it's not yet in use, despite mandates for HIT rollout with penalties for non-adopters - ed.] within and across care delivery organizations and health IT software vendors.
4. Discuss policy and process implications for deploying the Health IT Hazard Manager via different organizations (i.e., AHRQ; Office of the National Coordinator for Health IT; Patient Safety Organization(s); Accrediting bodies; IT entities).

In effect, HHS seems to be saying "we're working on the HIT risk problem, but roll it out anyway; if you get harmed or killed, tough luck."  This seems a form of negligence.

Have we thrown out all we know about medical research and human subjects protections in face of the magical powers and profits of computers in medicine?

-- SS