Showing posts with label NIST. Show all posts
Showing posts with label NIST. Show all posts

Wednesday, November 23, 2011

Two Opposing Views of EHR: InformaticMD vs. NextGen's Holder of "American Medical Informatics Certification for Health Information Technology"

The AMA's publication American Medical News recently quoted me following comments from IOM EHR Safety committee member Richard Cook in the Nov. 21, 2011 article "IOM calls for monitoring and probe of health IT hazards" by Kevin O'Reilly:

... Not everyone on the panel agreed with delaying FDA regulation. [Per the IOM report on health IT safety released Nov. 10, 2011, see here - ed.]

Committee member Richard I. Cook, MD, filed a dissent in the report in which he recommended that health IT systems be regulated as class III medical devices.

"It is quite remarkable that we're in this situation," said Dr. Cook, associate professor of anesthesia and critical care at the University of Chicago Pritzker School of Medicine. [Also, an expert in Medical Informatics - ed.] "It's not surprising that such adverse events are being found related to health IT, and it's not surprising that those promoting these systems have neither looked for them nor anticipated them. To make large-scale investments in these systems and only now be looking at the impact on patient safety borders on recklessness."

Scot M. Silverstein, MD, agreed.

"The bone I have to pick with the IOM report is that the action agenda is weak," said Dr. Silverstein, a consultant in medical informatics at the Drexel University College of Information Science and Technology in Pennsylvania.

It is unethical to expand health IT so dramatically without understanding the precise nature of the risks it poses to patients, Dr. Silverstein said.


Ironically, right below my statement was the following from HIT industry figure Charles Jarvis, blaming the user:

Users faulted

Leaders in the health IT industry also had their share of objections to some of the IOM panel's conclusions.

"We don't think there's a great deal of data to substantiate that there are major safety problems with the majority of electronic health records systems in use today," said Charlie Jarvis, executive committee vice chair of the EHR Assn., a trade group that represents 46 organizations that supply most of the EMR systems implemented in medical practices. "These products are safe, dependable, time-tested and display a lot of the safety features we think are necessary to prevent problems going forward."

Jarvis, also a vice president at the health IT firm NextGen, said vendors and the government should work to help physicians and other health professional users understand systems, take advantage of their safety features and avoid errors.

[Charitable translation: computers are infallible, so medical errors due to HIT are the user's fault, the Sept. 2011 National Institute of Standards and Technology (NIST) report on usability be damned. Clinicians should spend their valuable time learning to compensate for and then actually wading through mission hostile user experiences. If only those stupid doctors and nurses would use our cybernetic miracle tools the way we want, the members of the EHR Association could be making even more money. Oh, and by the way, the NIST's concept of "use error" [1] is nonsense. - ed.]


I presume the "EHR Assn." is the HIMSS EHR Association, with HIMSS itself being a gargantuan "cause-based, not-for-profit organization exclusively focused on providing global leadership for the optimal use of information technology and management systems for the betterment of healthcare":

The HIMSS Electronic Health Record (EHR) Association is a trade association of Electronic Health Record (EHR) companies, addressing national efforts to create interoperable EHRs in hospital and ambulatory care settings. The EHR Association operates on the premise that the rapid, widespread adoption of EHRs will help improve the quality of patient care as well as the productivity and sustainability of the healthcare system.

I observe that there are no conflicts of interest here that could cause Mr. Jarvis' stated opinions to be skewed towards the rights of computers and away from the rights of patients ... right?

First, Mr. Jarvis makes a logical error related to the error illustrated in my earlier post today "Magical Thinking on Health IT from ModernMedicine.com." His error is that of "proof by lack of evidence" [2]. No need to actually study the issue rigorously, despite repeated risk management-relevant incident reports (as opposed to the industry's preferred and highly erroneous term "anecdotes").

Just one recent, highly alarming example of an "anecdote" affecting probably tens of thousands of patients due to programming malpractice and grossly negligent quality assurance, at both vendor and end user hospitals, is illustrated here. Since it's an "anecdote", perhaps Mr. Jarvis would agree there's nothing to see there, so we should all move along.

(See the Aug. 2011 post "From a Senior Clinician Down Under: Anecdotes and Medicine, We are Actually Talking About Two Different Things" which puts the misuse of the "anecdotes" label in its proper place - the garbage can.)

Not only is "proof by lack of evidence" in the face of hair-raising incident and defects reports (e.g., as in FDA's MAUDE database) a prima facie logical fallacy unfitting in medicine, and in fact alien to medical ethics, but the IOM report specifically stated in no uncertain terms that nobody really knows the magnitude of the risks. This is due in part to numerous inhibitory factors in evidence diffusion. From the IOM report:

... Several reasons health IT–related safety data are lacking include the absence of measures and a central repository (or linkages among decentralized repositories) to collect, analyze, and act on information related to safety of this technology. Another impediment to gathering safety data is contractual barriers (e.g., nondisclosure, confidentiality clauses) that can prevent users from sharing information about health IT–related adverse events. These barriers limit users’ abilities to share knowledge of risk-prone user interfaces, for instance through screenshots and descriptions of potentially unsafe processes. In addition, some vendors include language in their sales contracts and escape responsibility for errors or defects in their software (i.e., “hold harmless clauses”). The committee believes these types of contractual restrictions limit transparency, which significantly contributes to the gaps in knowledge of health IT–related patient safety risks. These barriers to generating evidence pose unacceptable risks to safety.

Imagine such a situation in, say, the pharmaceutical, automotive, aviation, or nuclear power industries. The responsible individuals would likely be hauled off to jail.

However, this all may be irrelevant. After all, who can argue with the expert personal opinion of someone who holds "the American Medical Informatics Certification for Health Information Technology?" That astonishing credential could conceivably elevate Mr. Jarvis' opinion over all others - even mine, with my meager background in the domain.

I don't know if he still claims that credential, but he did as I described in my post about prior interactions with Mr. Jarvis and NextGen (dating back to 2004) in my Feb. 2009 post "NextGen and Vendor/Doctor Dialog: Yet Another Patronizing EHR Company of Certified HIT Experts?"

I guess the fact I'd never heard of such a qualification represents my dearth of familiarity with the field of Medical Informatics and healthcare information technology.

-- SS

Notes:

[1] “Use error” is a term used very specifically by NIST to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging, misuse of color-coding conventions, omission of information, etc. From "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).

[2] Example of proof by lack of evidence, courtesy Scott Adams: "I've never seen you drunk, so you must be one of those Amish people. "

More on these issues is at the site "Contemporary Issues in Medical Informatics: Common Examples of Healthcare Information Technology Difficulties."

Sunday, October 23, 2011

NIST on the EHR Mission Hostile User Experience: Blame the User? Nyet...

I have often had to respond to those who claim that EHR's don't cause medical errors, users do. That subset of the health IT irrationally exuberant seem common in the health IT industry and pundit channels.

NIST (The U.S. National Institute of Standards and Technology) has recently issued a draft report "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).

The report is quite negative concerning the low usability of today's commercial health IT, and recommends significant improvements that have been standard practice in other mission- and life-critical IT sectors for decades.

I will have more to write about this report, but I found a passage and footnote early in the report striking.

Of note, from the new NIST draft report:

This passage, from page 10:

The EUP (EHR usability protocol) emphasis should be on ensuring that necessary and sufficient usability validation and remediation has been conducted so that use error [3] is minimized.

[3] “Use error” is a term used very specifically to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging [or lack of messaging, e.g., no warnings of potentially dangerous actions - ed.], misuse of color-coding conventions, omission of information, etc.

This passage describes what I have termed a mission hostile user experience.

"Blame the user" as the default, reflex reply to clinical IT-related medical errors, and the "hold vendor harmless for defects" clauses that facilitate this excuse are now heading to the junkpile - in the clinic, hospital, and courtroom.

It is my hope that the the Wild West, free-for-all, cavalier, get out of jail free days of the health IT industry are coming to a close.

Hint to health IT industry: you may actually need to invest in people who know what they're doing, instead of hiring the cheapest labor possible. (See, for example, my Aug. 2010 post
"EPIC's outrageous recommendations on healthcare IT project staffing" for more on that issue.)

-- SS

Oct. 25, 2011 Addendum:

This comment
(#5, October 25th, 2011 at 12:38 pm) by "a practicing front-end designer and application developer" over at HIStalk is quite interesting. The writer points out the differences between designers and developers, and opines that:

If you’re going to improve vendor design, it has to begin with an internal commitment to value designers and what they contribute to the product development process. They can not be an afterthought. There are very few companies with this mindset. Almost all HIT companies are developer-driven, so the first thought, and one that is promoted in this report, is to turn developers into designers. This will not work! A developer and a designer require two fundamentally different skill sets that are not easily transposed. Developers are trained to think rationally and analytically; design requires empathy for the user.

In health IT, empathy for the patient as well, I might add.

-- SS