Watching the ODF - OOXML Debate

by Kurt Cagle

The debate between ODF and OOXML adopters even on these pages has definitely managed to clearly push people into one or the other camp, to the extent that I find it amusing to see the degree to which both sides have rallied around their respective flags. I'll freely admit that I am very much in favor of seeing ODF's acceptance as an ISO standard - it has, in most cases, respected the principles that I myself believe about standards, to whit:

13 Comments


2007-02-18 18:10:29
...Deliberate Incompleteness. Technically, what ODF has proposed is largely for the Word processing portion of ODF...


Incompleteness? It's not a bug, it's a feature.


Funny how ODF-partisans don't brag about this when they lobby governments to mandate ODF. They could say something like this: "ODF is the One True Format. Our spreadsheets are an ISO-approved open standard. Of course, the specification is incomplete, so ODF spreadsheets can't interoperate with different spreadsheet programs. But we designed it that way deliberately."

Kurt Cagle
2007-02-18 20:31:49
There is a fairly extensive portion in the ISO ODF standard that is devoted to spreadsheets. The issues that currently exist have to do primarily with the question of function sets and implementations thereof. The ODF standard is an XML one, thus the issue of which language to define such standards has been contentious. Yes, this is a controversial issue, and one that has been debated fairly heavily even within the ODF group itself.
Gary Edwards
2007-02-19 00:24:47
Hi Kurt,


Thanks for the honest appraisal. Concerning the OASIS ODF Open Formula Project, which began in October of 2004, the most difficult challenge David A. Wheeler's group has had to overcome has been that of harmonizing their multi spreadsheet application universal formula work with the un documented Microsoft Excel formulas. The long shadow of Excel continues to dominate discussions and proposals.


If Microsoft had documented the Excel formulas, or better yet, joined in the universal formulas effort, David's group would have been done long ago.


Like most things concerning Microsoft and XML, they withheld until the last minute anything having to do with encumbrances, standardization, documentation, and legal assurances. No doubt the advance of ODF has pushed them towards the halfway open point where OOXML sits today. And even that has been a tectonic struggle.


Microsoft continues to hedge, holding back until the last possible moment important documentation and disclosures. The Excel formulas are a perfect example of how they play the game, with every intention of maintaining their advantage and control over our digital future. Which is fine with me. Just don't ask ISO/IEC to carve that business objective into an international standard.


~ge~

M. David Peterson
2007-02-19 00:33:35
> The central crux of the current debate is, and should be, whether Microsoft's OOXML does in fact represent a standard that is conceivably implementable by anyone outside of Microsoft <


Kurt, we've already discussed this and we both know the answer: Yes.


Both ODF and EOOXML can be looked upon in many ways as componentized document formats in that they can each be implemented one piece at a time without requirement that the other pieces be in place to function properly. The problem isn't "can the entire spec be implemented" and instead "is each component complete enough such that they can be implemented *independently* of one another, and in a way that ensures that if an application claims to support a particular sub-format of the specification, that it does so in compliance with the specification."


It seems that people keep glossing over the interoperability aspect to all of this. If I save a word processing document in one application, can I open it in another and pick up where I left off without any loss in fidelity? And yes, the loss in fidelity is quite important, so I'm not sure I understand your argument to the (somewhat) contrary.


There is another aspect to this that everyone seems to have lost site of; Something you actually seem to touch on, but with a somewhat opposite conclusion: Formats are supposed to be containers that provides ways to ensure lossless communication of information. They shouldn't define the features of an application, and instead provide ways to ensure that if I open, edit, and save a document in one application, I can open, edit, and save that same document in another application which claims support for the format in question and expect it to just work without any loss of meaningful data.


That said: What defines meaningful data? If I have a specific border I prefer in my documents, is that important information to propagate from one application to the next? Some people would argue no, but many would argue yes!


What about information regarding application state? Is that meaningful data? Sure, and while it would be nice to ensure 1-to-1 fidelity of application state, unless a world filled with drone applications is something we are striving to achieve (we aren't, are we?), then application state doesn't even make sense in regards to ensuring 1-to-1 fidelity, *especially* when it is specific to things like user preferences in which don't apply to someone else who opens the same document, expecting for their own user preference be set, firmly in place.


> There is a fairly extensive portion in the ISO ODF standard that is devoted to spreadsheets.


No there's not. There is in the recently ratified-by-OASIS ODF:v.Next(), but that isn't an ISO standard (though no doubt that at some point in the future (a year?) it will be) as of yet, and at present time, there isn't enough information to properly implement a spreadsheet application > an application-type, btw, in which relies on a foundation of *formulas* to get it's job done. Can I define cell width, height, border color & size? Sure. Does it matter? Sure, but not from the standpoint of processing numbers and output the result, and instead from the standpoint of defining the containers in which the non-existent formulas are to exist.

M. David Peterson
2007-02-19 00:39:16
@Gary Edwards,


>> Thanks for the honest appraisal. Concerning the OASIS ODF Open Formula Project, which began in October of 2004, the most difficult challenge David A. Wheeler's group has had to overcome has been that of harmonizing their multi spreadsheet application universal formula work with the un documented Microsoft Excel formulas. The long shadow of Excel continues to dominate discussions and proposals. <<


So then compatibility with existing document formats is important? Huh, weird... I always get the impression from the Anti-EOOXML camp that compatibility with previous document formats isn't something that belongs in the specification, and is the job of the application itself.


It's good to see that there are some folks at least in the ODF camp who understand that compatibility with previous document formats is as important as it is. Because, in fact, as you point out, compatibility between previous document formats *IS IMPORTANT*.

M. David Peterson
2007-02-19 00:45:35
>> and by all indications it has not only alienated the ISO committee but also largely reduced its credibility in other standards areas at a time when open standardization is becoming recognized as the only legitimate standardization form. <<


According to who? From what I have gathered from various, *reliable* sources is that, in fact, the ISO committee hasn't been alienated in any, way, shape, or form, and in many ways, the exact opposite has been the result.


Where do you see that suggests otherwise?

Alex Brown
2007-02-19 07:56:14
>> They took a gamble that they could continue to use their weight to get their way, and by all indications it has not only alienated the ISO committee but also largely reduced its credibility in other standards areas <<


Not sure which committee you're referring to, but the technical committee charged with reviewing ODF, and lined-up for OOXML (JTC1 SC34 WG1) had virtually zero input into ODF - the input came from ISO member countries.


For OOXML, this committee has so far also had no involvement - ISO has been collecting initial comments from member countries.


If anything 'alienates' this ISO committee it's that these office format wars generate so much noise, while getting coverage for the XML standards work it does (producing RELAX NG, NVDL, etc.) is so difficult!



2007-02-19 09:04:10
The ODf specs are not fully patent unencumbered as you call it. It seems that Sun has waived patent rights only for versions of the standard that Sun is participating on. That could not be accpetable for a company depending on Office development like Microsoft
See also:
http://ooxmlhoaxes.blogspot.com/2007/02/ooxml-hoax-2-standard-is-not-really.html
Gary Edwards
2007-02-21 21:33:06
@M. David Peterson


So i guess i t would surprise you to know that compatibility with existing file formats was an issue for the OASIS Open Office XML TC dating back to the very first meeting, December 18th, 2002.


The minutes of this meeting have an interesting reference, "
- The TC agreed that transformability into potential Microsoft office XML formats could be sensible, but is not a formal requirement."


Since i was at that meeting, i can assure you that there was much more behind the statement than the minutes indicate. For one thing, at the time of the meeting there was no such thing as "Microsoft office XML formats".


The issue was that of being able to express perfectly an XML encoding of Microsoft office file formats - the infamous billions upon billions of over forty billions of Carl Sagon big B Billions of Microsoft binary documents. Could the Open Office XML specification be written to accomodate the XML transformation of those billions of binary documents?


So important was this concern that it was nearly made part of the Charter Objectives statement. Sun and a few others took the position that although it was a worthy and most important objective, referencing a particular vendors file format issues in such a way was inappropriate for an effort that sought to create a universal file format independent of applications, platforms or specific vendors. No vote was taken except that no one disagreed with the suggestion that the issue be punted down the road, and dealt with at the specification level. Much of this "dealing" happened in February of 2003 when section 1.5 of the Open Office XML specification was crafted at the first face to face meeting.


Also keep in mind that it was only through the persistent prodding of Novell's Jody Goldberg that Microsoft finally committed the Excel spreadsheet formulas to Ecma 376 - August of 2006 i think. Those formulas were a major dilemma for the ODF Open Formula group in that they truly sought to be a universal formula effort, complete with a clean clear harmonization with the legacy of billions upon billions of Excel spreadsheets.


Microsoft claims (wrongly i might add) that ODF is insufficient and unable to capture the advanced feature sets of MSOffice. Yet, it was their decision to not participate in the OASIS Open Office XML TC - even though they have been members of that same TC with "observer" status since it's founding in November of 2002.


So of course if they chose not to participate, and they chose not to reveal, document or disclose the blueprint to the billions of binary documents, one might think ODF would fall short on three critically important measures:


...... conversion fidelity (the billions of binaries)
......... round trip fidelity
............ application interoperability


The amazing thing is that ODF can do so well with the three measures even though Microsoft refused to cooperate in any way.


Most people underestimate how advanced the science of reverse engineering those billions of binary documents has gotten. My guess is that we can nail those binaries with a 95% conversion fidelity at the document level.


I say "document level" because as soon as you push the issue of application interop, the differentials between application feature sets and layout engine impedance rears it's ugly head, and mashes the truth of how good the reverse engineering conversion process really is.


The OpenDocument Foundation recently released the ACME 376 plugin for MSWord to prove this point.


Hope this helps,
~ge~

Gary Edwards
2007-02-21 22:03:50
It seems that people keep glossing over the interoperability aspect to all of this. If I save a word processing document in one application, can I open it in another and pick up where I left off without any loss in fidelity? And yes, the loss in fidelity is quite important, so I'm not sure I understand your argument to the (somewhat) contrary.


So, what kind of application interoperability does EOOXML provide? The Microsoft-CleverAge-Novell Translator Project produces crap. By the end of this week Novell is rumored to be releasing the OpenOffice.org version of the MCN Translator - with the same results - crap. Apparently "interoperability" between OOXML and ODF was not part of the Microsoft "design".


And then you have the brewing problem of the Ecma 376 eXtended file formats; Microsoft Office Open XML (MOOXML) and the binary InfoSet version of MOOXML now on display in MSOffice 2007 (Excel).


As to the complaints about ODF 1.0 lacking universal formulas, if Microsoft would have publicly released the Excel formulas or otherwise participated in the ODF process, they would have been in ODF 1.0.


And how about all those complaints the Ecma 376 didn't include a hint about the billions of billions of VB Macros so important to all the MSOffice bound business processes and line of business apps that couldn't move to ODF no matter how great the conversion fidelity or Ecma 376 <> ISO 26300 XSL Transformation process?


The ODF 1.2 work is very focused on application interoperability issues. This work is on line and can be found in the efforts of the three sub committees; formula, metadata and accessibility. There is also a number of harmonization efforts underway, most notably with the ODF <> UOF effort.


Given that the Ecma 376 fast track effort has been somewhat derailed, a number of proposals have already been floated to present ISO/IEC JTCS1 with a plan for harmonizing ODF <> OOXML. Maybe ISO/IEC can persuade Microsoft to work with their existing product, ISO 26300?


~ge~

Gary Edwards
2007-02-21 22:26:42
@Alex Brown


This is not true!


"Not sure which committee you're referring to, but the technical committee charged with reviewing ODF, and lined-up for OOXML (JTC1 SC34 WG1) had virtually zero input into ODF - the input came from ISO member countries."


In August of 2004, the EU approached the OASIS Open Office XML TC with a number of requests:


.... Support XForms (forms, data binding, custom-defined schemas)
....... Support SVG & SMiL
.... Change the name so Microsoft will no have an excuse not to use the file format. (OpenDocument was the fictitious name used in EU IDABC Valoris Report referencing a universal XML file format)
....... Submit Open Office XML - ODF to ISO/IEC for approval


By October of 2004, everything the EU asked for was in place, including the preparations for ISO/IEC. There was full six months of re editing and review of the ODF specification to meet the very strict and unforgiving ISO/IEC presentation and contradiction requirements.


James Mason, the co-chair of ISO/IEC JTC1 SC34 joined the OASIS ODF TC in September of 2004 specifically to assist in this difficult editing process. He has remained a member in good standing ever since.


Patrick Durusau is the other co-chairman of ISO/IEC JTC1 SC34. Patrick is one of the founding members and primary editors of ODF!


The entire ODF spec was reviewed and re written expressly to meet the requirements of ISO/IEC. Submission of ODF to the OASIS membership at large for consideration was withheld for six months while the ISO/IEC work was completed! We didn't want any ISO/IEC changes that would have caused us to have to resubmit ODF for OASIS approval! So every precaution was taken.


Compare this to our friends at Ecma 376. They left out an entire section of their submission to ISO/IEC. That alone should have disqualified Ecma 376 from fast track consideration.


The bottom line for Ecma 376 is that there is an existing ISO/IEC product, ISO 26300, that is the international standard for desktop office suite XML file formats. When ODF hit ISO/IEC, there was a gapping hole that could not be filled by RTF or PDF, or any other ISO product. That hole been filled, so the terms of assessing Ecma 376 are quite different than those faced by ODF when it was first presented to ISO. ISO/IEC rules are pretty clear about contradicting existing ISO products.


Let the harmonization begin,
~ge~

Kurt Cagle
2007-02-21 22:55:02
Gary,


I have heard, from a number of different sources, information corroborating everything you've said here. Real standards development of solid specifications is hard - witness the travails that SQL underwent for standardization in the 1980s, or many of the early W3C specs in the 1990s or SGML, for that matter, in the 1970s, and we've managed in the course of 40-50 years to develop a fairly thorough understanding of what it takes to achieve good standardization and what does not.


I see standards bodies as being very much analogous to patent granting agencies. A good patent system should by necessity be tough and difficult to gain patents from unless the work truly represents a significant innovation; such patents have authority, and such authority tends to engender efforts to continue pushing the envelope of invention.


Unfortunately, when a patent system gets suborned, and starts granting patents with inadequate due diligence or honest consideration for prior art (or worse, starts granting patents on the basis of corporate earnings or political campaign contributions) then patents become spurious, unfair, difficult to litigate and in the long run fairly worthless. The American patent system has been moving in this direction for awhile, to the extent that much of the rest of the world largely ignores software patents with impunity.


Organizations such as ECMA have gained authority largely due to their earlier work, but increasingly are being seen as "cheap patents" - akin to granting mail-order degrees. Microsoft has availed itself of the ECMA as their preferred standards body for some time largely because ECMA has generally had considerably lower "standards" for standards, and for a while such an arrangement worked both to Microsoft's and ECMA's benefit.


Increasingly, though, ECMA itself has lost clout because of the perception that it has not in fact provided sufficient rigour in the standards processes, and this has made it increasingly difficult for it to attract others who would see that organization as being truly representative of the great majority of users of these standards.


I suspect the process with OOXML may prove to be a costly one for ECMA in the long run, and I wouldn't be at all surprised that the upshot of this is that Microsoft will find it increasingly difficult from here forward to find a captive standards organization ... and perhaps will finally realize that they have to play well with others or they will lose credibility.

Rick Jelliffe
2007-02-21 23:19:19
Gary: Alex Brown is a member of WG1. He said *WG1* (proper) had basically no input or role at that stage. Invoking Jim or Patrick or the EU or Humpty Dumpty or anyone on SC34 is simply not responding to his point. And what is his point? To refuse to see the task of WG1 in terms of whether it makes OASIS or Ecma happy, no matter whether it then means you ODF guys then start an anti-ISO campaign (which seems to underly several recent comments I've seen around). WG1 has long had its own agenda: the fact that it primarily and historically serves the document processing community and is not impressed (in my personal understanding of the members, at least) by the economic or political imperitives of the large suite vendors, means that WG1 people don't like to be railroaded. Suite interoperability is important, but it is only a small indeed minor part of the document processing ecology that WG1 is attempting to foster.