If It's Not Broken, Don't Fix It!

by M. David Peterson

Update: Aristotle Pagaltzis has taken a few moments to provide some extended information on this topic. Please see his provided insite below.

Thanks Aristotle!
---

Update: Well well, it seems we may have an even more interesting twist to things than I had originally thought could be the case. Please see David Powell's counter arguments to the "obvious benefits" of the Thread Extension below.

Thanks David!
---

[MUSTREAD]
Update : After additional comments from Aristotle, who was then followed up by James Snell, author/editor of the Feed Thread Extension specification, I have come to the conclusion that the FTE is how I originally believed it to be. I've followed up their comments with a new post to *hopefully* bring this all to a close. [INSERT: Just reread my own follow-up in which I mentioned I would mark this update with a MUST READ attribute in regards to their comments. thus the added attribute above.]

It *IS* Broken... So It *NEEDS* Fixing!

Thanks to Aristotle, David, and James for providing their insite into this matter. It's a pretty important matter, and worthy of a good discussion... which is what just took place. Thanks folks!

[Orig. Post]
Dare Obasanjo aka Carnage4Life - Pointless Reinvention of the Wheel

Great, not only do we now have to deal with the an increase in the number of competing XML syndication formats due to the Atom process ...By the way, have you seen the Atom 0.3 vs. Atom 1.0 debate? I told you so... we now also have to deal with duplicates of all the popular RSS extensions as well? Give me a break!


This post started out simple, then grew into something completely different and ten times longer than its needs to be. There's some good content that can be salvaged for a separate post, on a completely different topic. In the mean time...

Unlike Dare (or maybe his point is just not to break existing software, leaving the existing extensions in place if they already exist, adding new extensions instead of replacing the old with the new?), I understand there is both the need and desire for the Feed Thread Extension.

But like Dare, I recognize that breaking software by replacing existing extensions with new ones causes more damage than it could EVER do good (unless I have completely missed the point, it seems to me this is what his gripe is about... if its not about replacing old with new, and instead never supporting the old, and choosing to implement the new, then I'm not so sure I understand where the problem is.)

Please don't do that. I REALLY like Atom. It's an important document specification that goes beyond personal preference over RSS 2.0, as it solves some very real problems that need solving. If significant and recognizable members of the Atom community start breaking the software that the Atom specification helps fix, the message sent is one of being less concerned with fixing problems, and more concerned with having things your own way.

I know for a fact that's not the message intended to be sent, but I'm pretty sure that's the message being received.

Thanks for your consideration in this matter. It seems like a pretty important one to me.

20 Comments

Aristotle Pagaltzis
2006-04-09 12:12:58
I think the Feed Thread Extension is far and away better than any of the specs it replaces.


And while I understand not wanting to break existing clients, if you want to take advantage of Feed Threads, known precedence problems arise when clients support all present extensions.


Of course, Dare is a client developer, and as such, he’s understandably going to be grumpy every time someone creates something new that his code will need to support… but I think in the case of Feed Threads, the intermittent breakage is worthwhile.


This is a multi-pronged issue, and you can’t really take any singular stance about it.

M. David Peterson
2006-04-09 17:27:37
Hey Aristotle,


Thanks for providing your insite. I recognize the fact multi-pronged issues can and easily do exist. And I DEFINITELY understand and agree... The Feed Thread Extension has been thought through from beginning to end, discussed in the community via public forums, and the result is that it provides a MUCH better overall interface for providing a way to communicate, and to keep track of that communication for a given topic/thread.


I'm not sure I know what all of the issues involved here are, and in fact I can say for certain that I don't based on the above post. While its probably enough to state "there's more here than meets the eye" if you have a chance and can provide any extended information, I'm sure quite a few folks, including myself, would appreciate the extended information provided.


Either way, thanks for taking the time to comment!

David Powell
2006-04-10 02:36:09
Why exactly is the Feed Thread Extension is better than its predecessors? Let's compare:


wfw:comment - provides a link to the comments for the entry.


FTE - provides a link to one or more sets of comments to an entry, these sets could be alternate representations of the same comments, or different sets of comments, you'll have to guess.


slash:comments - provides a count of the number of comments to an entry


FTE - provides a count of one or more sets of comments to an entry, problems as above. This count attribute doesn't use Atom extensibility points, so is liable to be dropped by Atom infrastructure (Windows Feed Platform is a concrete example), but apparently people won't mind the data-loss


wfw+slash - fits the Atom extensibility model, despite pre-dating it.


FTE - doesn't fit the Atom extensibility model, despite being designed around it.


To be fair, thr:in-reply-to is ok.


Does this all boil down to which approach has the fewest (boo! hiss!) namespaces? The XML community would get a lot more done if they spent more time thinking about interoperability, and less about subjective aesthetics, but I suspect that this is the risk of XML having no real-world model behind a document. In OO, RDBMs, and RDF people can have reasoned debates about design. XML design is like watching adults play with Fuzzy Felt.

M. David Peterson
2006-04-10 03:09:04
Hey David,


Lets see if I can stop laughing about the fuzzy felt comment long enought to follow-up your comment. :D


I must admit that the points you have brought out seem pretty painful to the "FTE is better" argument. Of course, I did see the tail end of the discussion on the Atom list, and it *seemed* to end with good vibes based on some last second adjustments based on community preferences of the usage of the @ref and @id attributes (if I'm remembering correctly... would need to check the archives to be certain as to if it was these two attributes for sure.) I guess maybe the good vibes were not as wide spread as I assumed.


I need to stop doing that (assuming :)


So then this conversation gets more interesting. And it seems to me that if there are problems with WTE, then maybe discussing them in the more wide open blog-based "forums" here on OReillyNet/XML.com will certainly allow for folks who would otherwise not participate in the mailing list to participate in a place that might feel a little more neutral. Not that I have anything wrong with the mailing list, but when you know who the folks are that your message is being sent to it can sometimes be a bit intimidating. Not the bad, intentional type intimidation, just the kind that makes you think twice about posting to the list.


Of course, this has never stopped me from sharing my own views on various matters, but there are times where I certainly wish it would... Such is life for a punk a$$ hacker with an attitude such as myself. :D


Ultimatelly my hope is that if problems exist, they simply get fixed. Life is simple when we live in bubbles.


But bubbles have a tendency to pop... Not always a good thing.


Thanks for your additional insite!


Anyone else care to take it from here?

Aristotle Pagaltzis
2006-04-10 11:05:39
Funny how David dismisses the one thing that makes the FTE better than extremely coarse wfw:commentRss in a single off-hand sentence. The real value proposition of the FTE is the thr:in-reply-to element.


wfw:commentRss allows you to associate a feed for comments with an entry. That means your comments must be a flat list, and it also means that you need to poll one feed per weblog post, so it’s infeasible to stay current with all threads on a weblog that has had hundreds of posts over a couple of years. Theoretically, you can implement nested comment threads with wfw:commentRss – if each of the posts in the comment feeds has a comment of its own. Can you say combinatorial explosion? I have often wondered what the guy who wrote that spec was thinking; how could anyone have missed the obvious scalability problem of that approach? Basically wfw:commentRss has exactly one use case.


In contrast, the FTE allows you to have a single comment feed that contains the replies to all of your entries while still associating each comment with the right entry – or, in the case of nested threads, with the right entry-or-comment. You can even put both your entries and your comments in a single feed. You can also continue doing the per-entry comment feed thing. Or you can do several of those at once. You can do it pretty much any way you want, because the granularity of thr:in-reply-to is entry-to-entry, not entry-to-feed.


Robert was vociferous about the thr:count attributes for the comment feed links, but if you consider that an entry may have any number of replies distributed among any number of resources, and any of these replies may appear in any number of these resources (a case which can be disambiguated using the Atom ID on the replies – this is Atom, remember?), then it makes a lot of sense. This thing allows for a lot more use cases than just blogs with flat comment list – you can go so far as to completely replicate the semantics of Usenet, distributed news feeds and all, which takes us full circle, but anyway, let’s not veer into that.


Okay, so in Dare’s post I just saw he mentioned annotate:reference. I had never heard of that one before. How many clients support that? And how much work will it be to convince any single client developer who has no support for any of these to support just the FTE (and all of it), vs convincing them to support slash plus wfw plus annotate (and then you’re still not covering all of the semantics the FTE allows) without quitting half-way through?


This is pointless reinvention on the same level on which Atom was pointless reinvention of RSS.

James Snell
2006-04-10 11:52:32
Heh, I had a response to David all worked out, then I refreshed the page and noticed that Aristotle beat me to it. The one additional comment I would make is that wfw:commentRss does not have a stable specification associated with it (sorry, a blog post or two does not count). For many potential implementors, a stable spec published via an organization with clearly defined IPR rules and community review processes is not just a convenience, it is a requirement.
M. David Peterson
2006-04-10 12:52:51
Aristotle,


I don't think I could argue against your points even if this was my all out intention (to argue for the sake of arguing)... which its not.


It all seems pretty straight forward to me that this is something that needs to happen.


From a personal perspective, there are two things I hope to see happen:


1) Atom become the primary data feed for anything that goes beyond just simple, one way syndication, and even then I think there are enough issues just with this portion of RSS 2.0 to warrant promoting Atom as the defacto base standard in which RSS feeds can implement atom:element* to fix the areas that are broken, without breaking those in which have built their brand against the RSS name (which I personally have my own feelings how to gain the best of both worlds in this regard, but its not something worth pushing as in the end its the quality of feed that matters most) There are certainly enough reasons and references to make this a pretty straight forward process.


2) Have the same group of folks that built Atom use the proven process of getting the right people into the right "room" in which anyone with internet access can join in (just like it is now) and build the necessary extensions that will cover the areas specifically and intentially left out of the Atom spec (like the Feed Thread portion) such that there can be one core base of primary extensions that cover a 95+% use case scenario such that within a years time the vendors can build in support for these extensions and feel confident that in doing so they won't be faced with problems down the road when its suddenly discovered "oh, this won't work for that, so now we need to break our apps again to implement a proper fix."


To me, the reason I attached myself to the use of Atom was because it was pulled together to fix problems by folks with a proven history of fixing problems and building great software.


If that same group is willing to employ their services, once again at no cost, to work on the extension areas that need work...


Well then its obvious to me its time for folks to simply get out of the way, and let them do their job.


I'll update this post, to mention both your comments, as well as James comments as MUST READ.


Thanks for adding all of this additional insite!

Aristotle Pagaltzis
2006-04-10 14:11:56
Heh, now if only that reply had been a little less embarrassingly incoherent.
M. David Peterson
2006-04-10 14:18:40
Made perfect sense to me... Or at least enough to gain a complete understanding as to the importance of FTE, so if it seemed incoherent (which to me it didn't), it was at least coherent enough to make sense as to why this was such an important extension... :D


Thanks again for helping me, and hopefully others as well, gain this understanding!

M. David Peterson
2006-04-10 16:01:42
James,


My apologies! I meant to add a follow-up to your comments to the same comment box as that in which I responded to Aristotle.


Your point regarding the importance of the specification process, as well as the final result of that process is absolutely spot on. It seems absolutely NUTTS to me when I hear people make comments that a well written, well developed specification is not something that is all that important. But, as made obvious in a post to my personal blog last August > http://www.xsltblog.com/archives/2005/08/standards_bodie_1.html < that's exactly the attitude that permeates the tech landscape by people who simply don't understand that a specification isn't something designed to bind, limit, or in other ways hold back the ability for any given entity (personal, corporate, sector, etc...) to make progress. As you point out:


>> For many potential implementors, a stable spec published via an organization with clearly defined IPR rules and community review processes is not just a convenience, it is a requirement <<


Real progress, growth, innovation, etc... comes when folks have the ability to minimize the risks involved with EXTENSIVE investment into new areas of technology. The way to minimize the risk, of course, comes in the form of a specification, in which they can use to build software, hardware, or other tangible items against a common interface which ensures a certain level of interop, and at very least enough common ground such that the overall costs of building the base foundation for these technologies can be "shared" across a larger base of manufactures.


Of course the long term reduction in cost of goods can be measured with much greater precision when well understood formulas can be implemented because of an understanding that "with every increase in production of X units, the overall cost of goods will be reduced by Y". With a specification, the increase of X is in FAR GREATER proportion than without.


With all of this said... I don't think I could agree with your point more than I already do. Of all the variables that go into the formula that quantifies the rate of potential progress in any given sector/society, the existence of a well written and designed specification in which contains input by ALL parties who plan to work against this spec is the final multiplier of a complex equation in which the lack of a specification is ~0, the presence of a specification [range 1-n] where n represents the commitment of parties to build against this same said spec.


Why people don't get this, I will NEVER understand!


Thanks for taking the time to add your additional (and obviously, authoritative) insite to the overall conversation!

David Powell
2006-04-10 16:37:22

The issue of whether or not it is good for interoperability to re-implement (and extend) the features of wfw+slash with FTE is interesting, but I am more interested in discussing the quality of FTE itself:


Why is replies implemented as an atom:link with non-standard attributes, when thr:in-reply-to is implemented as an extension element? Using non-standard attributes on atom:link is an interoperability issue that can be fixed simply by replacing the element with an extension element. I don't see any costs in doing this. Is atom:link used just because of subjective prettiness; does this trump interoperability? I refer back to my Fuzzy Felt quip.


I feel strongly about this. Using non-standard XML attributes on Atom elements is even more damaging to Atom, than QNames in content is to XML. mnot's quote: "Putting QNames into your XML content is like using TCP packets as delimiters in an application protocol.", describes the problem well. It binds Atom entries, feeds, links, and authors to the XML document that they were found in and prevents applications, APIs and frameworks from working with these concepts at a useful level of abstraction.


M. David Peterson
2006-04-10 17:26:29
Hey David, I don't disagree with your point, but I also don't know enough to accurately provide anything that goes beyond,


>> Anyone want to take this and run with it?


Hopefully someone will pick this up and we'll see where it goes from there.


I don't have the time to search the archives right at this moment, but if you happen to have a link or two in regards to this discussion I would certainly appreciate it as this seems like something that certainly justifies a deeper understanding as to the reasoning behind this particular decision, which to be honest I didn't realize was the case. (note: I need to read the spec again, but are the attributes in their own namespace? If yes, if not mistaken this is perfectly legal, if not looked upon as something to avoid if at all possible.)


That said, I would hesitate to suggest that the reasoning goes beyond something that could be considered reasonable, given the folks in whom I am aware of that were involved with the development. But as mentioned previously, I only witnessed the very tail end of the discussion, and in no way participated as I was too far out of the loop to be able to even comment, much less join in the debate.


Of course, (again, as previously mentioned) that hasn't been true in ALL cases... I tend not to worry too much about jumping in head first into any given conversation, but generally speaking this will take place only when the topic at hand it something that I do have a reasonable amount of experience with and/or overall understanding, even if the overall subject matter is part of an ongoing discussion in which I've arrived a bit late to gain full context, something thats bitten me more than enough times to suggest maybe I should avoid such things if at all possible. Then again, there are times when the context is the problem in and of itself, so the bites and the bites back tends to even themselves out over time.


What this has to with anything... to be honest... I'm not really sure. I guess I'll leave it, but this is obviously not something to connect with the overall conversation.

James Snell
2006-04-10 18:15:58
David,


Ironically, the primary reason for using link for "replies" is because it makes very little sense to duplicate the function of the link element. I considered a thr:replies element. Ultimately, it would have looked and acted darn near identical to the atom:link element (with attributes like type, href, hreflang, etc). One needs to consider whether or not it makes sense to introduce a new namespaced extension element that duplicates the basic function of atom:link just to support the addition of two *optional* and *purely advisory* attributes.


The way I read the spec, Atom has two levels of extensibility. The first is defined by Section 6 of RFC4287. The second is defined by XML. RFC4287 allows undefined attributes and undefined content on the link element. This means that unknown attributes and elements MAY appear but aren't covered by Section 6. Pure RFC4287 implementations may or may not choose to support such extensions. RFC4287 implementations that also claim to support the Feed Thread Extensions should likely be expected to support 'em. Publishers that choose to use FTE need to be aware that not all clients will be able to do anything with the thr:count and thr:when attributes. That's ok.

M. David Peterson
2006-04-10 18:59:52
Hey James,


While I have heard others suggest that adding namespaced attributes is something you should avoid, and instead use a completely new namespaced element, I have found that the arguments have leaned more towards "just cuz' I prefer to work with elements whos attributes all belong to the same namespace of the containg element" instead of performance related issues, or other equally valid concerns. While I can recognize that a it doesn't always look 'pleasing to the eye' this tends to be the case when someone chooses to implement a namespace extension that contradicts what would look pleasing all by itself on a plane white sheet of paper, much less contained within the opening and closing angle brackets of an element.


I'll leave plenty of room for someone who feels theres more to it than this to provide extended reasoning, but for now I would have to side with you on this as it takes less processing code to handle the additional two attributes to then use the existing atom:link code as is, than it would be to add in the extended code base to process the exact same thing -- which would, of course, require definining a link element as part of the FTE spec, which when you think about all of the constructs and rulesets that apply to atom:link, theres obviously a significant amount of overhead just to handle something thats already within scope of the containing atom document.


Of course, dependent on the processing language used, the actual amount of code needed to handle the addition of a namespaced element or attribute is not really all that much, but the spec piece of this alone is enough to justify this, regardless of the fact that the additional code necessary to handle the processing would be minimal.


I'd be interested to see if others have strong opinions that suggest that a namespaced attribute that differs from the namespace of its containing element holds more potential problems than I am allowing credit for, but if anyone does take up the challenge, instead of using theory, use code. My guess is that if the code suggests performance problems, between myself, and various 'friends of the family' we can find ways to fix that problem in a jiffy :D


Oh, and Javascript code that walks the DOM doesn't count. If this were to be your 'proof', your problem has nothing to do with namespaced attributes, and everything to do with your insistence that the code presented even qualifies as code, much less something that highlights the problems of using namespaced attributes.

M. David Peterson
2006-04-10 19:04:03
One other thing... In regards to the thr:count, I can see complete justification in this being optional given the fact that from the outside looking in, its more of a convenience than anything else, as you can just as easily count the comments and arrive at the same number. Regarding thr:when - I need to look at the spec again to be reminded what its used for. But if its suggested that its okay for it not to be present, I would have to think its probably for similar reasons as the thr:count where its more about convenience than functionality/capability.
M. David Peterson
2006-04-10 19:21:27
David,


Regarding the 'wfw+slash'; I realize that this may sound like a silly thing, but to me the use of the slash namespace is the problem. When you're working with production systems in a competitive marketplace, using a 'brand name' within a code base in which your customers both can and more than likely will actually look at from time to time, using 'brand name extensions', especially if you just so happen to consider slashdot a competitor, which given the primary focus of slashdot in general, can be just about anybody they decide to trash-n-bash on any given day. Of course, just as much praise can be found their as well, but my guess is that there are enough folks that have been slammed by the slashdot crew to suggest that in and of itself is going to cause people to frown upon its usage in a production system that their name is attached to.


Again, this probably sounds a bit ridiculous, but my guess is the reality is probably something close to this, if not spot on for a lot of different companies out there.


NOTE: To be honest, I cant even say for sure if the slash extension is related to slashdot, although it seems to me that last time I looked through the various docs the connection was obvious. Even still, theres an association thats made regards of any actual connection. So unfortunately, no matter how you look at it, the adoption rate as web feeds continue to move even further away from the early adopting slashdot crowd, is going to decrease for seemingly silly, yet very real reasons.

David Powell
2006-04-12 16:59:31
Hi David,


Forgetting the specifics of FTE for a moment, I'll just answer your question, and explain my problems with namespaced attributes in Atom:



In Mark Nottingham's post about the XML Infoset, he describes 3 approaches to using XML. I strongly prefer staying away from the complexity of the Infoset (approach #1), and instead basing applications around a reusable model that can be converted to and from XML (approach #2). This way people can work with Atom entries in terms of Java classes, database tables, or whatever, rather than always be required to work with the XML. One of my goals in participating in the Atom WG, was to ensure that the spec was clear enough that Atom (including extensions) could be represented in some form other than Atom XML without any data loss.


Atom doesn't require a document to be valid according to a schema or DTD, and in any case, foreign namespaced attributes are specifically allowed on any atom: elements by the RNG, they are just "undefined".


Namespaced elements on the other hand are regarded as extension points to Atom: properties of the feed, entry, or person depending on where they appear.


When you start adding undefined attributes to an Atom document, you get something that can only be represented in XML, and can't be represented in the simpler entry/feed abstraction anymore. So you basically prevent everyone from working with nice Atom APIs, and from storing these attributes in their database schema. The abstraction is made to leak, and everybody suffers.



Enough of the theory: imagine an Atom consumer (such as an Atom Protocol server, or an Atom parsing API), that stores entries in Java classes, database tables, a legacy CMS, or anything else that isn't the exact XML file that the entry was received as. A basic implementation might be expected to do some sort of competent job of preserving core Atom properties such as atom:title, and atom:content; a better implementation might preserve some specific extension elements that it knows of; a better implementation might generically preserve extension elements as blocks of XML; but it is really asking a lot for the implementation to preserve the XML document identically to the one that was received.


Windows Feed Platform is a real-life example of an Atom implementation that does try to preserve core elements, and extension elements, but makes no attempt to preserve undefined mark-up, such as foreign namespaced attributes.


Atom doesn't actually define conformance levels for preservation of extensions, so implementations are permitted to do whatever they want, but realistically implementations will preserve Atom mark-up with varying levels of fidelity. The less fidelity that is required to preserve the extension, the more interoperable the extension will be. It would be nice if every APP client, APP server, Atom feed, Atom intermediary, Atom parsing API, Atom storage backend, and Atom aggregator supported every extension as soon as it was conceived, but this "boil the ocean" approach isn't going to happen. I think extensions are an important part of Atom, so the alternative: having as many bits of the infrastructure as possible, provide some generic support for arbitrary extension elements (just as most of the HTTP infrastructure provides generic support for arbitrary HTTP headers), is a better option.



M. David Peterson
2006-04-12 17:33:58
Hey David,


I am away from my office at the moment (@coffee after catching up on some errands real quick) so I didn't see the notification regarding your comment, and am just noticing this now after logging in to the system to write Yet Another Follow-Up based on the Atom syntax mailing list conversation that I just finished reading.


I had actually been "writing" the post in my head while I was running my errands based on your most recent post, only to discover that Aristotle's follow-up which only strengthened what I plan to write. I *HOPE* to keep it short and simple. We'll see how I do :D


I'll post the link to the list when it's ready.


Thanks for the follow-up!

Mike Macgirvin
2006-05-27 09:12:28
I guess I'm late to the party here, but my question is why thr:in-reply-to is a new element and why a link wasn't used for this as well (?) It walks like a link relation, it talks like a link relation. It adds a 'source' attribute, which seems a bit dubious to me since the related item is already given. New relations may be requested of the IANA. Couldn't comments be completely provided by link relations? OK, we'd need an extension for the thr:count and thr:updated perhaps, but couldn't these be an edge case? It seems to be a needless cluttering of the XML tagspace. We've got link relations already. Comments/replies are fundamentally link relations. This look like an effort to turn them into kitchen sinks.
tramdol
2007-01-25 23:56:29
I always have terrible trouble with comment-related plugins that require me to put some line in the comment loop; I can never seem to find the right spot. Can anyone tell me where I should put the php line in my comments loop? I haven not modified anything much, and I would be very grateful. Thanks!