Response to Critical Safety Issue for the PCEHR

Feb 7, 2012

While I was on leave at Tamboon Inlet (and completely off the grid), Eric Browne made a post strongly critical of CDA on his blog:

I contend that it is nigh on impossible with the current HL7 CDA design, to build sufficient checks into the e-health system to ensure these sorts of errors won’t occur with real data, or to detect mismatch errors between the two parts of the documents once they have been sent to other providers or lodged in PCEHR repositories.

Eric’s key issue is that

One major problem with HL7 CDA, as currently specified for the PCEHR, is that data can be supplied simultaneously in two distinct, yet disconnected forms – one which is “human-readable”, narrative text displayable to a patient or clinician in a browser  panel;  the other comprising highly structured  and coded clinical “entries” destined for later computer processing.

It’s odd to hear the central design tenant of CDA described as a “major problem with CDA”. I think this betrays a fundamental misunderstanding of what CDA is, and why it exists. These misunderstandings were echoed in a number of the comments. CDA is built around the notion of a the twin forms - a human presentation, and a computer processible version. Given this, it’s an obvious issue about how the two relate to each other, and I spend at least an hour discussing this every time I do a CDA tutorial.

Eric complains that clinicians have no way to inspect how the data and the narrative relate, nor is there an algorithm to test this:

However, the critical part of the document containing the structured, computer-processable data upon which decision support  is to be based is totally opaque to clinicians, and cannot be readily viewed or checked in any meaningful way

This is true - and it would be a whole lot more concerning except that this is true of all the forms of exchange we currently have - it’s just that they don’t have any human fall back to act as a fail safe check. Of course, in a perfect world, this wouldn’t be necessary. The data elements would be clearly and unambiguously defined, everyone would agree with them, no one would use anything extra, and all the implementations would be perfect. This is not the world we live in - it’s a pure governance fantasy, but one that some of Eric’s commenters share:

I can’t imagine going into any true IT organisation and proposing storing the same information constructed in two different ways in the same document, and with no computable requirement to have them synchronised (Andrew Patterson)

My initial response was, no, of course not. But in actual fact, this is ubiquitous in health care, and CDA is designed for the reality that we have. Note that CDA is designed for exchange, for the cracks between systems that cannot or might not agree on data fields. People might not like that, but the PCEHR is very much living between the cracks of the existing clinical systems, unless we replace all of them now.

CDA itself doesn’t have much to say about the relationship between the data and the text. It implies that there must be one, but because CDA covers such a wide variety of use cases, CDA itself doesn’t make the rules; instead, the rules are delegated to CDA implementation guides to make comment about this. And a number of the NEHTA implementation guides do exactly that in response to the same concerns Eric expresses.

Back to Eric’s concerns:

Each clinician is expected to attest the validity of any document prior to sharing it with other healthcare providers, consumers or systems, and she can do so by viewing the HTML rendition of the “human-readable” part of the document… However, the critical part of the document containing the structured, computer-processable data upon which decision support  is to be based is totally opaque to clinicians, and cannot be readily viewed or checked in any meaningful way.

Where as now, with HL7 v2, they can’t see it, and can’t attest the validity at all. Instead, they must trust their systems, and there is no level of human to human fall back position at all. With CDA, they still must trust their systems, because they still can’t see the data portion - that is no different. But they also have a level of human to human communication that doesn’t exist with v2. CDA solves this problem, but does not solve the fact that we still have to trust the systems to exchange the data correctly to get computable outcomes (aside: I’m far from convinced that the clinical community wants more than a modicum of computable outcomes at the moment).

Of course, this still leaves the question of whether the data and the narrative agrees with each or not. The important thing that you have to consider in this regard is how do you build a document? How would you actually build a document that contains narrative and data that disagree with each other? Once you start pursuing this question, it becomes clear that a system **or clinician **that produces CDA documents that disagree between narrative and data have a serious underlying problem. Note the emphasis on clinician there. In a genuine clinical system producing genuine clinical documents, the system can’t prevent clinicians form producing incoherent documents - it’s up to the clinician. It seems to me, from watching the debate, that whether you think that is good depends on whether you’re a clinician or not.

I’ll illustrate this by describing two scenarios.

  1. A (e.g. pathology) system produces CDA entirely from structured data. The document is produced in the background with no human interaction. In this case, how can the narrative and the data disagree? Well, if a programmer or a user misunderstood the definitions or intended/actual usage of the data items.
  2. A (e.g. GP) system generates a CDA document using user selected available data from the patient record using a clinician defined template, loads the section narratives with their underlying data into an editor, and lets the clinician edit the narrative (usually in order to add additional details or clarifications not found in the structured data). In this case, the narrative can disagree from the data if the user updates the document to disagree with their own data.

In either case, there is an underlying problem that would not be detectable at the end-point were only the data provided. CDA can’t solve these problems - but the fact that CDA contains both narrative and text doesn’t create them either

Finally, the actual usefulness of containing narrative and data is unexpectedly illustrated by Eric’s own post, in an example where he appears to think that he’s criticising the presence of both narrative and data.

As an illustration of the sort of problems  we might see arising, I proffer the following. I looked at 6 sample discharge summary CDA documents  provided by the National E-health Transition Authority recently. Each discharge summary looked fine when the human-readable part was displayed in a browser, yet unbeknownst to any clinician that might do the same, buried in the computer-processable part, I found that each patient was dead at the time of discharge. One patient had been flagged as having died on the day they had been born – 25 years prior to the date that they were purportedly discharged from hospital! Fortunately this was just test, not “live” data.

Firstly, Eric shouldn’t have used technical examples provided to illustrate syntax to application developers as if they are also semantically meaningful (most NEHTA examples aren’t due to time constraints, though I’ve done one or two - it’s a lot slower than you think to produce really meaningful examples). But the date of death is actually buried in the portion of CDA that is data only, not narrative. And because Eric chose the wrong stylesheet (not the NEHTA one), his system didn’t know about the date of death, and ignored it. Had CDA actually contained a narrative portion for the header too, this would not have been a problem. Which brings me back to my earlier point: in the world we live in, not everyone shares the same set of data and processes them with no errors etc.

CDA isn’t a perfect specification - nothing is (subject of a series of posts to come) - and it does have it’s own complexity. But the problems aren’t due to containing both narrative and data. Eric says:

I know of no software anywhere in the world that can compare the two distinct parts of these electronic documents to reassure the clinician that what is being sent in the highly structured and coded part matches the simple, narrative part of the document to which they attest. This is due almost entirely to the excessive complexity and design of the current HL7 CDA standard.

This I completely disagree with. The inability to automatically determine whether what is being sent in the highly structured part matches the narrative is not “entirely due” to the complexity of CDA, but is almost entirely due to the problem itself.

Finally, Eric says that

NEHTA should provide an application, or an algorithm,  that allows users to decode and view all the hidden, coded clinical contents of any of the PCEHR electronic document types, so that those contents can be compared with the human-readable part of the document.

Actually, this is a pretty good idea, though I don’t know whether NEHTA can or not (on practical grounds). I guess that we should be able to, since the Implementation Guides will increasingly make rules about what the narrative must do, and we already provide examples based on rendering the structured data in the narrative. But my own experience trying to interpret AS 4700.1 HL7 v2 messages (Australian Diagnostic Reports) suggests that a canonical rendering application is even more necessary for that - but who could define such a beast?