Web Services, Weblogs and the Future.

by Timothy Appnel

In recent weeks a significant amount of discussion has been ongoing as to the future of Weblog APIs. At issue is that there are two similar, but different Web service APIs in use -- the Blogger and MetaWeblog APIs. Within each of those APIs are various interoperability and implementation issues and even some extensions. The community clearly wants one tool-agnostic API that all can utilize and integrate tools with, but there is differing views as to how this will and should happen.



The discussion began with a review of the existing Weblog APIs on Diego Doval's site that lead to a long thread of comments and follow-up posts. Adriaan Tijsseling creator of the Kung-Log Weblog authoring client offered his thoughts on the matter having worked extensively with numerous implementations available. Most notable in the recent posts is the one by Blogger's Evan Williams where he explains how their API came to be and why Blogger has not supported the MetaWeblog API. He agrees with the call for a universal blogging API and adds that no one vendor control it. He concludes "I perhaps now understand the need for standards bodies more than I ever have before even though the term gives me willies."



I think that such an universal interface would be a great for the weblogging community and beyond. There absolutely should be one interface that we all can utilize and rely on. (It shouldn't necessarily be limited to just weblogging publishing though.) And yes, standards bodies give me the willies also.



This all being said, I'm left with some nagging questions and omissions in the design and implementation discussion that could effect the utility and effectiveness of such an interface -- international character support, robust extensibility and cohesion with RSS.



The most important amongst these issues is XML-RPC's lack of international character support. Adriaan Tijsseling noted to me that Apple's WebServicesCore and MovableType's API support UTF8 in XML-RPC, but according to the XML-RPC specification strings are limited to ASCII thereby undermining international character support. There have been numerous threads in the past on this issue such as this one made by Charles Cook.



To a lesser extent, but still significant, is robust extensibility. As weblogging expands and evolves, feature sets will become more diverse and feature sets in tools will begin to vary and hybrids emerge. What effect will this have on interoperability? What XML-RPC
lacks is a straight-forward and reliable way for it to be extended when warranted -- whether that's between two weblogs or two million. In comparison to straight XML, a XML-RPC struct is not straight-forward -- its quite verbose. Structs in XML-RPC don't have the benefit of utilizing namespaces and thereby cannot directly leverage previous work such as Dublin Core. Back in January, Sam Ruby published "Evolution of the Weblog APIs" that amongst many things, highlights this issue by comparing the implementation of services with XML-RPC and other formats.



The simple answer to both these issues would seem to be that the XML-RPC specification should be modified to address these issues, however XML-RPC has been declared frozen and not subject to change.



So I wonder aloud, is a broad API based on XML-RPC the way to go forward? This is a difficult question to definitively answer given the existing landscape, but given the circumstances one worth of consideration.



Furthermore, RSS and Weblog APIs are working with the same data -- why are two very different formats needed? As I have asserted in the past, RSS is a Web service we already have. RSS is even more widely deployed then either of these Weblog APIs. It can handle all international character sets and with the introduction of modules in 1.0 and copied in 2.0 you have all the extensibility built-in you need. RSS over HTTP (perhaps even wrapped in a simple SOAP envelope) may be a better long-term solution in terms of extensibility, simplicity, and international support. Merging syndication and APIs in this space seems a plausible and worthy consideration. This notion puts even more emphasis on the need to clean-up and better define RSS.



These are a difficult and contentious issues without easy answers. Nevertheless they are better addressed now rather then later in serving the long-term good of the community.



What do you think is the future of Weblog/Publishing APIs?


13 Comments

anonymous2
2003-05-23 14:11:19
XML-RPC battles
It seems to me that people want something that is as easy to implement and use as XML-RPC, but they want it extensible and UTF-aware. The original author of XML-RPC spec has declared his unwillingness to revise the spec.


Does this situation strike anyone else as being remarkably parallel to the circumstances surrounding RSS?


In the end, people are going to make extensions to XML-RPC whether Winer likes it or not. The only question at that point is what to call the result -- XML-RPC 2.0, XML-RPCng, or something entirely different.


Standards that become superceded by advances in technology are no longer standard. They become evolutionary dead-ends. Standards that evolve with the changing needs of users last longer.


In other words, feel free to get over yourselves, and just do the right thing.

kbedell
2003-05-23 19:45:11
Nice post and analysis
Standards aren't all bad - for example, look at all the Internet standards (HTTP, SMTP, DNS, etc.). These were all developed by ad hoc groups of individuals and honed through peer review. And they work great - they're the reason the Internet is what it is today.


So called 'standards' that are published by corporations or individuals with agendas to push or axes to grind *are* bad.


So the key to 'good' standards seems to me to be in the structure of the group(s) that develop them and the process of peer review they follow.


Peer review and acceptance is the critical point - which is why Internet standards are all *still* called Requests For Comments (RFC's) years after they are complete.


Though, to be fair, many of the RFC's were crafted by corporations or commercial working groups and then 'proposed' as standards. One that comes to mind is Netscape's proposal for HTTP-session management using cookies - years later it's still in use without substantial change.


anonymous2
2003-05-25 02:00:23
Use SOAP
SOAP is a great service, supported by all the big players.
anonymous2
2003-05-26 11:59:18
Use SOAP
SOAP is a useless waste of time and a wonderful example of NeedlessComplexity. Use HTTP. What else needs to be done besides GETting data, POSTing new data, PUTing changed data and DELETEing dead data? Focus on clean resource-space identifiers [URIs] and meaningful XML blocks. Then, DO NOT wrap them in an XML-RPC or SOAP noise. Just use the data. Focus on getting the existing tools [Mozilla, IE, Opera] to deal with authentication and POST/PUT correctly, rather than creating a whole set of tools. We have a good protocol [http] and a simple way to describe data-languages [xml] -- let's use them.
anonymous2
2003-05-26 15:35:34
XML-RPC battles
Dear Anonymous, go forth and innovate. Just don't call it XML-RPC and everyone should be happy.


To Timothy, dive in a little deeper into the MetaWeblog API. It is based on RSS 2.0. It's exactly what you want. Or put it another way, what more do you want?


Dave Winer

tima
2003-05-26 16:59:04
XML-RPC battles
I would begin with the first point in my post -- international character support. How would you provide it in a frozen specification that explicitly states strings to be ASCII?
anonymous2
2003-05-26 21:58:55
RSS has many problems too
A discussion recently on RSS for weblogs has sparked some concern over the huge variety in RSS standards. Check out some of the discussion at Six Log:


http://www.sixapart.com/log/2003/05/a_proposal_rss_.shtml

anonymous2
2003-05-27 18:48:41
XML-RPC battles
Read the archive of the XML-RPC mail list. This issue comes up regularly. I get tired of doing repeat performances. XML-RPC is so deployed it just can't be changed, not by anyone. If you do a little digging you'll see how other smart people have dealt with this issue. I've given my opinion more than once. Dave
tima
2003-05-28 01:23:22
Smart implementation of international character support?
I'm aware of these workarounds. As I've noted there are a number of different solutions including breaking from the specification to support UTF8. I would be curious to hear what users working with languages that don't fit into ASCII think.


There are different degrees of smart. Smart workarounds. Yes. Smart implementation of international character support. No.


"XML-RPC is so deployed it just can't be changed, not by anyone."


This is at the heart of my point. If XML-RPC cannot be evolved then indeed it is not the right thing for a publishing API to go forward with.

mentata
2003-05-28 10:20:34
Is RSS really a standard?
I'll admit up front that I have only recently begun looking at this, but I just spent yesterday looking at RSS feeds for a contracted project and I will attest that among the hundred or so I reviewed almost no two feeds look alike unless they are from the same source. Reading about the history of RSS, there is clearly some contention over how it's used, what it's for, and even what it's called. I've got a new translation for the acronym: Relatively Splintered Standard.


That doesn't mean I don't like what people are doing or that I wouldn't continue to invest, but the word "standard" means something different to me than what I've found. FWIW, as a programmer I would favor the RDF version that Meerkat uses because it gives me more metadata in parsing. However, I should also note that the current O'Reilly weblog feed only gives me articles from 2001.

anonymous2
2003-05-28 18:38:31
Use SOAP
Agreed on SOAP. Simple it Ain't.


On HTTP, you have to handle types (is is a date? Is it an integer? Is it a string?) on the server, and you'd have to know what it is, because 06116 is a zip code, a string, not an integer, so for every item of data, you'd have to send the type too, as in Val=06116&Type=String. But what if you're sending 3 values in a row? Which type applies to which value? or do you number them? as:


val1=06116&type1=string&val2=04&type2=integer&val3=Connecticut&type3=string


for:
ZipCode 06116
StateCode 04
StateName Connecticut


Then you end up with a parsing nightmare, with arrays, and how large are the arrays... So on and so forth.


Trust me, for structs, it would not be pretty.


anonymous2
2003-05-28 18:44:30
XML-RPC battles
I'll agree there.


I took a good look at it once.


I had to ask myself what XML-RPC really was.


Basically, XML-RPC is a Remote Procedure Call.


You pass variables, you get variables back, or an error.

tima
2003-05-29 07:42:04
SOAP w/ RSS could be simpler then you think.
I understand where you are coming from here and would suggest an approach that would allow for both.


Creating a SOAP/RSS hybrid would simpler then most people think when document literal encoding is applied. (Most are familar with the RPC encoding which does have more overhead and issues particularly in this case.) Sam demonstrates that at the end of his essay. http://www.intertwingly.net/stories/2003/01/26/evolve.html#SOAP


Here is some additional reading on the matter:
http://www.intertwingly.net/blog/1112.html
http://www.intertwingly.net/blog/1010.html
http://www.snellspace.com/blog/archives/000127.html


SOAP does have its place. Wouldn't it be helpful to post to your weblog via an email message? (This assumes the client is handling the SOAP encoding of course.)