XML Efficiency, Alternatives

by Jeff Bone

In various discussions I've been having about whether XML sucks and / or whether there are better alternatives for what it does, I've repeatedly eluded to something I read at some point about some research going on at IBM Almaden on the efficiency of XML encoding.

Well, I haven't exactly come across the original reference, but I've found some related research at IBM Almaden. The Vinci Service-Oriented Architecture includes something called XTalk, a loosely-coupled component bus that works through XML document exchange.

XTalk doesn't actually use XML per se, but rather a pre-parsed binary representation of XML. In the words of the authors:

"While the exchange presented above is shown in XML notation, the communication between Vinci components is not "pure" XML, but rather a semi-parsed, pseudo-binary representation of an XML document we call XTalk. This representation is used because most existing XML parsers are too expensive, in terms of code size, processing time and memory footprint, for use in interactive applications. Section 4 shows how this feature allows for Vinci to perform on par with optimized RPC implementations, and an order of magnitude faster than SOAP-based services. (Concerns about SOAP performance have been expressed before].)

"...To summarize, the main advantages of XTalk over XML are speed, size, and simplicity. Speed-wise, we have found our XTalk parsers to provide at worst a 3 times speed up over a hand-optimized, bare-bones XML parser. In practice, when compared to full-blown XML parsers such as Xerces, the speedup is closer to 10 times or more. Size-wise, our XTalk parsers are approximately two orders of magnitude smaller than comparable XML parsers, and memory footprints a a factor of 4 times smaller. For example, in PalmOS, our basic client, server, VinciFrame document model and XTalk conversion library has a size of only 13K."

So perhaps there's the 10x that was lurking in my head: a 10x speedup over stock XML parsers using a type-lite binary encoding using symbol tables. Now, information density / encoding efficiency has been studied and addressed in other efforts (WBXML, Millau, etc.) --- I'll keep looking for a formal treatment of that. Back to XTalk... there are trade-offs to its approach:

"None of these advantages are completely without cost. Our XTalk interpreters offer no type checking and conversion, well-formedness checking, or ambiguity resolution that are features of many fully compliant XML parsers. However, our view is that these features are typically unnecessary in a deployed application. Documents are necessarily well-formed and unambigous when constructed through programmatic document models instead of by hand. And should document validation or type-checking be a concern, Vinci allows it to be plugged in as a meta-service when and where as needed..."

The theory behind these compromises lies in the intent of XTalk. It is designed for an Intranet environment where developers work in a more coordinated fashion to integrate their services and components, and where security and other considerations are less than on the open Internet. The idea is that the messages need not be as strongly-typed or self-descriptive, since the usual use of any given message is strictly in some usual exchange of information. This is a legitimate engineering trade-off.

Point is, though, this isn't a safe set of assumptions in *all* the contexts XML is used in and all the functions it is designed to fulfill. If you can go with implicit typing, possible formedness issues, ambiguity, etc. --- i.e., if you can punt many common error cases or handle them yourself --- then maybe XML does such. Maybe something like XTalk - a kind of XML-lite with whatever representational syntax you want - maybe that's for you.

It's interesting to note that XTalk's transmission format is translated to an in-memory document representation based on FramerD, one of the classic frame data models. This lends some weight to the arguments of those who claim "frames are all you need."

This all comes back to the basic point I was making in the first article [4] on this topic. The operative philosophy in question is: this is a technical question, not a philosophical one. (Whew! Get that? ;-) This is sort of like strong typing in programming languages: if you need it in your particular situation, you really need it, but if you don't it's a pain in the ass. Similarly, if you need strong typing, extensible / nestable metadata, namespaces, etc. in a representational format, XML probably isn't too much better or worse than any alternative you could come up with. OTOH, if all you need is frames and / or you can make certain other simplifying / limiting assumptions, then XML --- and certainly XML with its full acronym soup brought to bear --- is probably overkill.