Articles Weblogs Books School Short Cuts Podcasts  

JXTA Search: A look at the future of searching
Pages: 1, 2, 3

Rael: Let's talk a little about JXTA and Web Services.

Steve: It's funny talking about the phrase "Web services" because the Web has so many connotations. I've been trying to figure out a better term -- something like "distributed services" or "network services," sort of agnostic as to whether it's the Web or Gnutella or JXTA. It doesn't really matter. The underlying issues there are: Can I support this thing in transport or does somebody have to pop through the transport for me to get into the other network? Can I understand the same messaging?

Rael: I've actually floated about the term "network services," too. It's a double-edged sword, in the sense that "net services" are usually associated with things like NFS, DNS, NNTP, etc., so it can be a little overloaded. On the other hands, it should be fine in terms of, are we inventing some of the new net services? Will NFS, DNS, messaging, presence management be the new group of Net services?

Kelly: There's a sort of irony of some of these different terms is they start to get overloaded pretty fast. I mean even "peer to peer" -- I tend to think about it more now, you know, as a framework for enabling easy communication between things.

Get cracking on your JXTA code

Announcing the Project JXTA Developer Contest. Enter your hot JXTA code and win a pass to the O'Reilly P2P and Web Services Conference in Washington, D.C., this September, and a Yopy PDA. Deadline is July 15.

Gene: Just to describe a little bit of our protocol. Essentially we have three kinds of messages. We have the request message, and we have a response message format, and we have a registration message format. And the request and responses are pretty clear. They're either coming from a client into a hub, or it could be a client into a provider. And the response is either coming from a provider to the client directly or from provider to the hub and then from the hub to the client and so on.

The registration component is interesting. You can compare this with a publish and subscribe model, if you're looking for analogies in the broader distributed network space. Each node registers with the hub -- although the registration actually fits on the node itself and so it could be registering it with a client, for all it knows. And the registration describes the metadata about that node, what it wishes to publish to the network about the content of its most frequently accessed database fields.

Those concepts describe metadata. In other words, some summary of the textual content. But alternatively you could think about it and our resolver -- which is the indexing component of the hub, the indexing and matching component of the hub -- the way it does the indexing is it indexes not only the textual content but also the XML tags that surround that textual content.

So you can play around with the idea that instead of throwing a query into our network and expecting a response, you could throw, for example, an XML-RPC request. You could say, "I want this request to be answered by somebody," and then the hub would look at that request and it would say, "Well, which providers that I know about first of all have even registered for this XML structure, and secondly have either explicitly or just as a wildcard registered for a particular method?" And then it would get routed to different providers around the network who could qualify that; the idea there is you can build this distributed, RPC mechanism without the clients or those other providers having to know exactly who's who.

Kelly: I just wonder where things are going next with JXTA Search.

Steve: A good question. Where JXTA's going next is we're giving it to you guys. We're outsourcing it. And the thing that we tried to do with JXTA Search is provide first of all an official searching framework for JXTA, and secondly try to demonstrate to people the advantages of having open and common standards for things like querying and responding. It's pretty amazing to me that we've got this far on the Web and we don't really have any common formats for this stuff. And as a result it's very difficult to build applications, and what we'd like to do is help people build these applications on top of the JXTA Search protocol, or a protocol that evolves from that. And we'd like to work with the community to improve it and make it a bigger, better, more successful thing, hopefully.

As you're probably aware, we're only planning to run a demo. We're not trying to sort of be a search aggregator in our plans with Sun. We're in business and JXTA's sort of creating some cool new technologies for this community.

Rael: How is JXTA Search integrated both team-wise and concept-wise with JXTA? How integrated is the experience for somebody who wants to build on JXTA and use JXTA Search?

Steve: You know, we've made no secrets about that our backgrounds not being in JXTA. As an independent company, it was impossible for us to be in JXTA, especially since it was evolving as we were evolving.

In terms of how we integrated it technically, what we've done is the existing hub service, the router and resolver, like I said, we designed them to be agnostic to the network transfer protocol and also the messaging format -- so essentially what we did was we adapted what we had and built a JXTA version of the resolver and the router that communicated out of a pipe using JXTA messages instead of using the HTTP post connection that we were using on the Web. As a result we've got a system that's pretty easy to use from both JXTA and also from a Web programming standpoint. I mean, we're not trying to push this angle but maybe a side effect of this is that people can see how easy it is to integrate these different environments such as JXTA on the Web.

Rael: You should obviously see folks building gateways to HTTP or SMTP or whatever vs. a pure JXTA search.

Steve: Yeah, absolutely. And I think that the key thing, as long as you're sharing the same messages -- you know, in our sense we send the same messages to a JXTA peer that we send to a Web server -- as long as they can understand what we're sending them, and as long as we have some method of bridging the different protocol bases, the transfer protocol bases, then you get to go.

The issue is in some of the design of the hub. In the HTTP version of the router, we have to be careful about how we manage HTTP connections, whereas in the JXTA version, it's actually a little bit easier because we can just connect and send and forget about it, and as long as we tag the queries and the responses, we can keep track of what's going on.

In some of these JXTA searches, asynchronous communication and grouping work very well when you want to do things like chaining lots of different hubs together so you don't have to keep all these synchronous connections open, whereas other situations that we have on the Web work equally as well. So on the one hand there's a sort of convergence of all these different ideas and different protocol spaces and messaging spaces, and at the same time I imagine that people will tend to keep what they know works while at the same time hopefully adopting cool ideas at JXTA.

Rael: Are there going to be other APIs that you're going to actually be focusing on, such as SOAP or XML-RPC, particularly for the Web services stuff?

Gene: You know, I think pretty clearly that's something that is on the map of things to do because the idea for InfraSearch is to make search something that's effectively protocol-independent, right? When you distribute search, you have the capability to make your search not only protocol-independent but also independent of the data format and of the data-storage paradigm. That's one way of looking at what InfraSearch does. And so clearly talking to JXTA Search nodes that are enabled using XML-RPC through a bridge or something like that is something that we would probably want to do.

Really, the problem that JXTA Search solves is this problem where traditional Web-based searches expected data to be represented on the Web in a very specific way on very specific types of systems, and using specifically HTTP and HTML, and that's proven to not be enough.

Rael: What you're really getting at here is resource discovery.

Gene: Right.

Rael: And the resources are a schema and you already hinted about RDF descriptions, but you're talking basically about distributed resource discovery. How would this relate to UDDI and the like?

Gene: Oh, the .NET question. I was waiting for that.

Rael: I may say "Hailstorm" in a moment, but --

Related Articles:

Learning the JXTA Shell

Hello JXTA!

The JuxtaNet

JXTA Takes Its Position

JXTA Chat, Sans Server

OpenNap Use Crashes

Gnutella: Alive, Well, and Changing Fast

Gnutella and the Transient Web

Gene: I think what will happen is that in the long term these things will play out, and in some cases UDDI will prove to be a successful approach. It certainly seems to be an effective way for people to easily publish information about their business, which is what seems to be the major use for it, like a global contact book. And I can imagine that getting extended as people start to use more Web services as to a way to publish what you've got available.

And I think the difference with what we've done is that -- and I'm not fully familiar with all the work of UDDI -- but I think the big difference is that we don't have any real structured definition as to how you should work with our network. You know, we have these envelopes and protocols that allow you to send, receive, and register what you've got, but then internal to that you can pretty much do whatever you want, as long as people are willing to accept queries or accept requests for that XML or specifically the data within the XML, and as long as they can be formatted and responded to OK.

So to answer your question in a long way, I don't see them as particularly competitive, and I think this is just going to play out. I think ours is a more attractive scheme for what we were talking about, the ad hoc distributed resource discovery networking, and so on.

Kelly: I wonder what the technical metrics for success for JXTA Search are.

Steve: I don't know, Kelly. I mean actually people like it, think it's a good approach, and I guess the measure of success if you open source something is that people both use it and extend it. And so I guess looking a year out I'd be happy if people were using it, if there were JXTA Search hubs all around the place, if providers were adapted to JXTA Search, if people were using it across their distributed peer-to-peer networks as well as on the deep search of the Web. And that people were making it better. I guess that would make me happy.