NullFlavor follow up

May 17, 2011

The last post explains NullFlavor. It’s not a short explanation. But incomplete data quality is like that. Half the explanation is about the problem, and half about the solution. NullFlavor is controversial. The biggest critic is Tom Beale of openEHR (Tom’s a regular reader of the blog. I look forward to a long comment below ;-)). And there’s certainly problems with the way HL7 does it in v3.

Note that there’s no actually much difference in outcome between the way openEHR handles nullFlavor and the way v3 does it - it’s more a matter of philosophy. The only actual model differences are:

  • The use of nullFlavors on interval boundaries
  • v3 allows individual values inside collections to have nullFlavors (but also puts more useful values inside collections)

With regard to philosophy, there’s a lot more difference. Tom won’t put nullFlavor on the base data type because that completely changes the nature of the base data types, and makes them unsuitable for use in a system. It certainly makes them harder to use, but HL7 takes the attitude that this is a matter of semantics - so you just have to do it, and anyway, HL7 is modeling data for interoperability, not for systems. (But this is a head-in-the-sand approach - the datatypes bleed straight into system design, and all sorts of people are trying to use ISO 21090 for system design - which is why Tom and I will eventually get around to publishing a “system implementation profile” for ISO 21090 dealing with issues like this for system design).

openEHR puts the nullFlavor on the element, side by side to it’s data. The intent is similar. If you tried doing this in HL7 v3, it would look real weird, and implementers would hate it more than they hate the current approach (by a factor of lots). The nullFlavor list is much restricted in openEHR - only dealing with missing data. Incomplete data is dealt with in different ways.

I think the openEHR approach works well for EHRs. Not so much for v3. I wouldn’t like to see a v3 where we did things that way. Not that I’m real comfortable with how we’ve done things in v3. If I ever did v3 over again, I’d really like to put nullFlavor up against the wall - I just don’t know if I would be able to.

In v2, we didn’t have a systematic approach to nullFlavor. There’s been several requests to add it in - one serious project proposal which consumed considerable committee time. But it’s just too big a change to introduce it in a standard where we are committed to backwards compatibility.

Really, in a perfect world, nullFlavor wouldn’t be required. But it isn’t a perfect world. And our models sure ain’t perfect.