Addressing fragments in REST

by Simon St. Laurent

REST offers a great way to build simple applications that Create, Read, Update, and Delete resources. But what if you want to get at part of a resource?


15 Comments

Fraser Goffin
2008-02-24 01:50:59
Some XML database implementations support 'well known' query string
variable names. For example eXist provides ?_query=... where ...
contains an XQuery expression or the location of an XQuery resource. XML databases increasing support a native REST interface too (eXist does) or you can use your preferred framework such as Restlet or Rails.
George
2008-02-25 04:57:05
RESTful way to work with XML Schema is something to ponder. Speaking of Ruby on Rails, XML Schema scaffolding and using REST for maintaining state sounds great. In my industry, the latest HL7 version 3 standards really heavily on XML. Thus, if we could work with XML like we do with databases, then life would be good.


len
2008-02-25 13:34:51
It works as long as you have a format that has a processor for a notation in which a given address type works. The problem is all of the formats for which a different notation requires a different process even if the address type can be mapped to it.


The endless arguments about this a decade ago stopped from fatigue. The web designers punted it away and documents that cannot be linked to using sub-resource links don't have them. Lazily, it evolved into 'send them the whole document and if they don't have a processor, they get an error from the client saying there is no registered handler'. Then they move on.


The other side is that some addresses can be declared for the published or on the wire version that won't work in a running instance. For example, you can use an id to point to an object in a real-time 3D scene graph, but using other DOM methods are not likely to work because such scenes being real time are in flux inserting and deleting objects in real time.


So properly, these ideas have to be scoped or you are doing Hytime again. I know you don't want that. ;-)

Simon St.Laurent
2008-02-25 14:00:52
I don't think reinventing HyTime is necessary, though yes, those problems will resurface if you try to solve too many problems simultaneously.


One of the nice things about the ?XML-XPTR query notation is that it could be a "well known" query string, and it also identifies a fair amount about the content being identified. It's likely to be applied to an XML document, using XPointer notation to identify the fragment.


It probably helps that I'm not expecting these things to survive in independent contexts. If there isn't a server available to run this query against, you're just not going to get anything - period. If a document is in flux, you're probably going to get an error message. That's pretty much what happens now, and the world hasn't collapsed as a result.


I don't expect to be "doing HyTime again". Working with XML is just complicated enough to reveal corner cases, but not so complicated that the corner cases usually matter. That's probably why a lot of people are using XML, while HyTime is a memory for specialists.

Subbu Allamaraju
2008-02-25 14:02:20
>> The problem is that developers can’t address the resource on which they want to work with sufficient granularity given their current set of tools and agreements.


You are right. However, I don't think using xpath such as "#xpath1(//book/chapter/title)" in URIs is desirable for addressing partial resources. Such a URI design ties the URIs to a particular type of representation (i.e. XML in your case), and goes against content-negotiation.

Simon St.Laurent
2008-02-25 14:15:34
"Such a URI design ties the URIs to a particular type of representation (i.e. XML in your case), and goes against content-negotiation."


Yep. It has to. Unfortunately, you can't really edit pieces of something unless you know how the pieces are defined.


Fragment identifiers are by their nature MIME-type dependent, which is why I like the ?XML-XPTR approach. The first piece says what your expected type context is (XML), and the second piece says what kind of fragment identifier you're using (XPointer).


(No, I don't think writing ?application/xml-XPointer would be an improvement.)


Effectively, though it may be redundant with other headers, this approach makes it extremely clear what kind of document you expect the fragment identifier to grapple with.


And it has to be clear, or there's no point whatsoever in bothering with this.

Subbu Allamaraju
2008-02-25 16:03:33
>> Yep. It has to. Unfortunately, you can't really edit pieces of something unless you know how the pieces are defined.


The client needs to know what it is changing, but it should be possible to express that independently of the types of representations possible for the resource, preferably by relying on a Content-Type for the PATCH request and not the URI. IMHO, URIs are content-type should be treated as independent axis for dealing with various representations of a given resource.


Sincerely,
Subbu

Simon St.Laurent
2008-02-25 18:33:05
"The client needs to know what it is changing, but it should be possible to express that independently of the types of representations possible for the resource, preferably by relying on a Content-Type for the PATCH request and not the URI."


First, I'm not using PATCH here. I think I made it pretty clear that I think PATCH is an extremely bad idea. "Red flashing lights" and all that.


Second, I don't think it's actually possible to express requests at the necessary level of granularity without some understanding of what it is I'm actually address. You just can't get ahold of anything much useful.


Now maybe it's conceivable that I could use a Content-Type header to say that I want this to apply to the XML and not an HTML, SVG, or JPEG representation - but I don't think that as a matter of practice that I should be using the same resource identifier for sub-resources in different formats.


I'm much more comfortable breaking the rules for URIs and content-negotiation than I am in breaking the rules for how REST works, if you really want a straight answer. Fragment identifiers have been a nightmare corner in the URI conversation for a very long time now, and I don't think that's ever likely to change.

len
2008-02-25 19:46:58
"I don't expect to be "doing HyTime again". Working with XML is just complicated enough to reveal corner cases, but not so complicated that the corner cases usually matter. That's probably why a lot of people are using XML, while HyTime is a memory for specialists"


That's a lazy dodge, Simon. The means to address subresources in markup are part of Hytime. What Hytime does is to go beyond markup and consider the problem of locator addressing in non-markup formats. The problem isn't XML. That's easy. The problem is exactly what is suggested in the follow ons: content types. Without format knowledge, no locator is reliable. The link can be but not the locator. This is the sense in which the URL/URI is uniform. The locator or resource representation is still required, thus content types or NOTATIONS.


Yes, just do XML. But one favor: if other standards for notations insist on having their own locator types, then the test is not where they conform to the XML sense of these, but where they are behaviorally consistent with the REST verbs. Yes?


BTW: adressing into a real-time 3D scene graph will not return errors if the locators ignore XML locations and rely on the namespace. It is dynamic. Time is a necessary component of the address.

Simon St.Laurent
2008-02-26 05:33:51
"That's a lazy dodge"?


Recognizing that HyTime went off a cliff because of its enormous and effectively infinite scope, and trying to avoid that fate, is a lazy dodge?


Remember, the Web itself was this useless little toy that couldn't possibly do anything relative to the much more powerful facilities that HyTime had. And even XLink vanished while the Web's mediocre linking soldiered on, ever stronger.


The rest of your comment is issues I've already addressed. I don't think it's sane to attempt to address fragments of something that's in constant motion, unless it has an underlying addressable structure that isn't in motion.


I certainly wouldn't object to other types with addressable components working along similar lines for their own content either. JSON seems like a good first case.

len
2008-02-26 06:05:19
"Recognizing that HyTime went off a cliff because of its enormous and effectively infinite scope, and trying to avoid that fate, is a lazy dodge?"


Actually, yes. Your view won't change and I won't attempt to change it here. I simply am pointing out that the only way to make your proposal logical is to scope it properly.


Your title is "Adressing Fragments in REST". Your content "Addressing Fragments in XML Using REST Verbs". So once again, if other standards for notations insist on having their own locator types, then the test is not where they conform to the XML sense of these, but where they are behaviorally consistent with the REST verbs. Yes?


In other words, formats beyond or different from XML devised for their own processor requirements may in fact use locator types not described in your proposals and still work with REST. REST isn't about XML linking and location systems. These are separable, yes?

Simon St.Laurent
2008-02-26 06:38:00
Len - you've got to start somewhere, and with something solveable.


I lack HyTime's ambitions, and frankly think the lesson of HyTime is that solving the full set of problems they addressed requires a God-like level of knowledge that simply isn't possible for now.


As I've said repeatedly, I'd be happy to see this approach reused for other formats, but I think trying to solve the problem for all formats is simply never going to happen. If the developers of other formats want to go in another REST-compatible direction instead, great for them.


Let's solve granularity one format at a time. The size, shape, and specification of the granules vary from format to format - there's no getting around that.

len
2008-02-26 08:50:00
You misunderstand. I am not saying use Hytime. Hytime can't be implemented politically. It requires too comprehensive an agreement which experience shows can't be achieved on the web. There are too many preconditions that have to be in place. But it is useful to understand the technical preconditions and leave the politics of the early days of the web aside. It was a naive design. Naive designs work as long as the implementor ambitions are similarl limited. That doesn't mean they don't change or the naive system doesn't work. But it is smart to know exactly where it runs out of steam. If that weren't the case, you wouldn't be resurrecting these topics or asking for a ten year review.


I use the example because it is the standard where the necessity of having locator types as distinct from links is made most clear. REST skirts that issue. You are trying to solve link/locators for XML, a problem solved by other unpopular standards in the past (XLink, XPointer, both derived from Hytime). What is different this time?


1. Frank admission that this is for XML only. Some locator types won't work for some flavors of HTML. Some won't work for PDF. Knowing in advance which locator types are supported by the specific implementation of a handler for a specific format is required to use these reliably. REST does not guarantee any of that for sub-resources. It can't. That is why all the discussions of 'what is a resource?' are ratholes.


2. Establishing the preconditions any format must meet to work with the locator types you are proposing or resurrecting.


BTW: if an XML 2.0 removed the requirement to have a root, how does that affect your locator types?

Simon St.Laurent
2008-02-26 09:24:15
To your point 1, I've repeatedly discussed how my solution is for XML only. I'd be happy to see the same approach used for other solutions, but that isn't remotely the same as denying that this is for XML.


To your point 2, I don't know what preconditions you're talking about, as I've already made it clear that this is for XML.


Removing the root element wouldn't bother me in the slightest. XPath can already deal with document fragments.


As to the big question of what's different this time, it's pretty simple: I'm not trying to do very much. It seems very clear at this point that I'm not solving the problems you find interesting - and frankly, solving the problems you keep bringing up seems like a remarkably awful idea.


I'm certainly NOT trying to "solve link/locators" for anything - I'm trying to assemble fragments of failed solutions into something that works for specific use cases. That seems like a valuable enough project to me, and one that won't wander into the weeds you keep describing.

len
2008-02-26 13:56:21
Best of luck then, Simon. I look forward to seeing how this works out.