XML Cowboys

Nov 1, 2011

While I was in Singapore recently, I spoke about the lessons I’ve learned through my various experience in Healthcare Interoperability. Much of what I spoke about has already been posted to my blog before, but one thnig I spoke about was “XML Cowboys”, which had an unexpected resonance in Singapore. “XML Cowboys” is a phrase I used to hear a fair bit at HL7 about 5 years ago, but it seems to have fallen out of use. I suspect that most of the cowboys grew up and either used standards or made their own up. Fundamentally, the phrase refers to implementors who look at the standards that exist, throw their arms in the sky, and ask why they have to be saddled with such complexity from a bunch of committees who clearly don’t understand the real world. “And anyway,” they say, “we have XML now, which is self describing, so let’s just use XML.”

I understand this impulse - standards are complex because:

  • They’re designed by committee (both their greatest strength and weakness)
  • They are general case solutions in order to foster re-use of the standard
  • The cater for all the difficult cases that have arisen over the years

But most of all, they’re complex because there’s nothing so complicated as someone else’s solution to your problem.

I dipped my own oar back in the XML Cowboy seas back then too, for all those reasons. Standards are a complicated way to solve things, and it’s a lot easier to just do something direct. Well, it’s easier in the short term. And that’s the key - if your success measures are all short term, the pressure to avoid standards can be overwhelming, because the benefits of standards don’t accrue in the short term.

This is something I see a lot, btw - that people don’t understand the value proposition of standards - you give something up here, in order to gain over there. Here and there might be separated by time, or be different communities, but there’s always a separation. And then people act all surprised when a group of people who are doing all the giving up don’t appear to be very committed to the process.

Well, an integration project with short term deadlines and no long term perspective is going to have a terrible time justifying not shifting the complexity into the future (see the 2nd law of interoperability). Because that’s all you can do. In my experience, XML cowboys don’t understand two fundamental things:

  • XML is not self-describing - it needs dictionaries, schemas, agreements to give it meaning. And the agreement is the expensive bit - it takes $$$ to maintain agreement over time and multiple teams
  • All that complexity they reject - too much of it really exists, but they’re not sophisticated enough to know that it’s there. Their project will only succeed if they can get away with pushing that complexity over the horizon

Given enough time, XML cowboys learn the value of agreement and forward planning, and start to appreciate the things that the standards offer.

But the pressure to just do it in a custom way and avoid engaging standards never goes away - and it won’t, as long as it’s possible to incur technical debt without accounting for it. The harder question to figure out is when that’s actually justified, as opposed to occurring through ignorance.

Note: if you scale this up, you get to national programs, and engaging with standards means dealing with their process complexity too. My next post will be about this.