Open Source Java: No magic pixie dust

by Diego Doval

This Thursday, at JavaOne, there will be a panel titled "The Big Question." The JavaOne page on the topic offers the following (cryptic) description:
Numerous individuals and organizations suggest that JavaTM technology adopt a new community and development model. This panel will dive into the tangle of granular technical and legal issues, including the potential tradeoff between technologist's calls for openness vs. the market's demand for compatibility. Panelists include leading technologists, experts on legal and copyright issues, top industry analysts, and mainstream users of the technology.
So is "a new community and development model" codespeak for open source? Even if the panel description doesn't make it clear, that's what everyone is expecting.


Open sourcing Java has been a hot topic in the community (particularly in Java-oriented weblogs) during the last few weeks. Most people seem to be for it, and I'd say that there's been as much community pressure as anyone can imagine on Sun to do it.


There has been a lot of argument over whether Sun would gain or lose with this, but little over how Java, or the developers that work with it, such as myself, would benefit from such a move. Note for example, open source notable Eric Raymond's Let Java Go open-letter to Scott McNealy, where he makes several points on how open sourcing Java would benefit Sun (to the point of making a much-disputed stock price comparison), but very little on how this would be good for Java. As a contrast, columns like this one by Dan Gillmor with more focus on Java itself are few and far between.


So, there must be an implicit belief that open sourcing Java would "be good", or would suddenly bring droves of developers into the Java world (how exactly would this happen is unclear--presumably because now that Java is "free" they would feel comfortable using it?), or would suddenly "fix" whenever is a problem for each of us, and so on. I'll come back at the end to some of the advantages that I think open source would have for Java, given the right context, but my point is that these will vary from person to person, and yet there is little mention of them--and how can that be possible unless every proponent assumes that it will solve their problems?


What comes to mind is a quote from Jamie Zawinski from the Cathedral and the Bazaar:

[...] in 1999 one of [Mozilla]'s principals [Jamie Zawinski] caused a bit of a sensation by resigning, complaining of poor management and missed opportunities. "Open source," he correctly observed, "is not magic pixie dust."
And yet, from all the buzz (dare I say hype?) surrounding open source and Java, one could be forgiven for thinking otherwise.


Eclipse


But why this belief? In my opinion, Eclipse has had a lot to do with it --as has Jakarta-- but Eclipse in particular is the shining star of the Java world right now. Look! A great platform! Open Source! Blasting barriers! Fast turnaround! Quality code! This is what Sun should do!


But we forget some things. Like the three or four dozen full-time developers that IBM devoted to this project. For a couple of years.


Or the fact that versions keep being pushed out against arbitrary deadlines. Not that deadlines are bad, necessarily, but as Don has noted it is possible that products that depend on Eclipse needed the release on a certain schedule, and maybe even for it to be ready for JavaOne. The many delays in the Linux Kernel come to mind here--delays for increased quality and because its development is less centrally managed.


Or the forty million dollars that IBM put into it.


Or the fact that several other companies are providing substantial amounts of money and resources to the project, post-IBM-spinoff.


(Btw, is all of this open source, Linux-style? Really? How many of its unique elements, such as the process, are present here?).


In any case, my point is not that Eclipse is "bad", but rather that open sourcing to any degree an already existing platform takes time, a lot of money (if you want to bootstrap it quickly), and direction. Even so, Eclipse's development has been as opaque to many people as Sun's for core Java. Nearing the end of the "Mx" releases there was a change on the look and feel, which many people complained about--because it caught most outsiders by surprise. And they couldn't really "go in and change it". Why? Complexity, lack of documentation, etc.


So if the JDK/JRE is just open sourced, maybe IBM or Oracle or BEA can benefit. But is that the point of open source?


Or rather, to go back to my original question, why is open source good for Java?


The problem with Java: direction, or lack thereof


In my opinion many Java developers that advocate open sourcing it are simply frustrated with Sun's management of it. (At least, I know I am). This doesn't mean, again, that an open source Java would be a bad idea, I'm just saying that maybe it's being pushed for the wrong reasons, and consequently doing it would deflect attention from the real problem for a while, not solve it, and maybe even exacerbate it.


By "Sun's management of Java" I am referring to their focus. Sun deserves a lot of credit for its stewardship of Java, but it's always been mildly schizophrenic about it. First it was all about thin-clients. Then it was all about server-side. Then it was actually about web services. Then it was about mobiles. And so on. In themselves these shifts are not necessarily a problem; what's a problem is the sudden nearly constant change of focus on certain APIs and not others. Year-in, year-out, APIs are suddenly elevated to high-priority status and then nearly abandoned (the case with JavaSound), or kept in life support. Lately, Sun has put a lot of work into Java Desktop System integration of the JDK, while ignoring pressing platform problems such as proper desktop integration, installation systems, size reduction, multiple JRE management, modularization, or HTML parsing/display support.


Or, rather, not ignoring, but doing something worse, "open sourcing" them.


Case in point is JDIC which provides desktop-integration capabilities that are little more than a band-aid considering where Java should be today. But what's the solution? "It's open source!"


Which I invariably read as: "we don't care enough about this technology to make it a priority, so we'll push it out in the open and see if anybody works on it".


But open source or not, Java needs some refocusing on the components that are missing from the platform: desktop integration through APIs that are properly ported and supported on the three major OSes (Linux, OS X and Windows), including installers, properties, web browsing, and so on. A clear plan regarding current APIs, including core Java APIs:

  • If the API is still evolving, what's the goal, what's missing, when can we expect it will be done, and how can the community contribute? (Open source would be ideal to inject energy into this process)
  • If, on the other hand, the API is in maintenance mode, can the process for fixes and requests be clarified? Here open source becomes almost crucial in the long run, as without it, minor bugs in rarely used APIs might remain unfixed through several cycles.
  • For new, or proposed APIs, do we still need JSRs, or can we provide a simpler, better mechanism, like what Apache does?
In this context, it is clear that an open source Java would be valuable. But without clear direction, and commitment, from Sun and others to drive the platform, guide it, and keep it from splintering, open sourcing Java could potentially create even more confusion.


Because open source is not magic pixie dust.


10 Comments

jwenting
2004-06-29 00:25:03
the problem with open source
The main problem with open source is a complete lack of accountability.
Anyone and his cat could (and in many cases would) just release an incompatible version of the JDK and JRE which would do precisely what the Java antagonists led by ESR want: destroy the platform.


This is seen in Linux as it is. DLL hell which Windows had finally all but solved with Windows 95 is back with a vengeance as dozens of incompatible versions of shared libraries float around the world and different applications require different versions which they install all neatly over whatever there is already thus disabling other applications or even core OS functions.
The OS itself changes in ways which break existing functionality without so much as warning even in between minor versions.


And I'm not even mentioning the sometimes crippling compatibility problems between Linux distributions which make the OS wars of the 1980s look tame in comparison (at least back then they didn't claim to be the same...).


Java as it is is open enough. Maybe the JCP needs to be able to act faster, but no more.

ilyanov
2004-06-29 06:50:11
An Honorable stand
To be honest my own feeling about OSS is mixed. I whole heartedly agree with you about the rationale about opening up Java even further. I am not entirely sure who exactly its going to serve. If one follows the old American addage about not fixing something if it aint broken, then there really isn't any reason to opening up.
However, I do hope you will accept my point of view about the value of Sun's stewardship.
Java does have a direction and singular purpose and I think Sun has been religous about it from day one.

Its the WORA axiom. I can transition from a java card to a cell phone to a smp server to a mainframe to a desktop to where ever a jvm is installed with very little effort. It is a mind numbing abstraction this jvm. My enduring weakness is that I have never been one for specialization in terms of API, actually I have never been much for specialization at all. I am at home moving from API set to API set, jvm to jvm. With Java I feel right at home because it allows me to experience different types of problems. The experience has been quite rewarding because I have had the opportunity to recognize patterns which are domain (vertical as well as horizontal) agnostic.



The other factor one has to keep in mind is the cambrian explosion IT has under gone since the Mosaic came out. Java's lack of direction is symptomatic of the industry in general. Take a look at XML specification or Web services, you will see a simillar situation. Direction did not exist since there was no road, no eco-system, Sun and allies built one up from the ground.


Finally one should also keep in mind the competitive pressures Sun is under vis-a-viz Microsoft and IBM. I believe Sun has done an admirable job of maintaining developer loyalty to the platform by offering something for every one. Sure we all have our pet peeves (I am a Jini man myself) but I do think the "majority" of java developers are quite content with their little pond or neck of the woods.


Anyway, the general thrust of your argument is valid and I hope it gets the coverage it deserves.
dscotson
2004-06-29 08:08:14
Open-Source worse than abandonment?
Quote: "Or, rather, not ignoring, but doing something worse, "open sourcing" them."


You eventually come round to saying that open-source would be a good thing as long as your other concerns are taken care of. So I'm confused about how Sun letting various non-core elements of it's Java platform bit-rot through neglect can be considered a better outcome than allowing outside developers with a vested interest in the code to continue development.


Or do the scare-quotes around "open sourcing" mean you think a genuine will for Sun to take advantage of the community is needed to see improvemnts?

diegod
2004-06-29 08:25:26
Open-Source worse than abandonment?
The quotes on "open sourcing" in that particular sentence refer to my view that in general Sun doesn't really open source those packages, but rather dumps them in the community with little if any guidance. As I said, in my view open source needs support structures, and even nurturing. Creating open source projects "just because" doesn't really do any good, and in particular when some of those components are (should be) really core elements, that is, Sun appears to open source things it'd like to have a foothold on but without serious investment. Both Mozilla and Eclipse took a long time until they established themselves (so did Apache, Linux, and others for that matter). I understand that Sun might be trying to do the best it can, but the lack of follow up with some of these projects leads to confusion rather than advances, since they are generally competing with similar OSS projects that already do what they plan to do (for example, most if not all the JDIC functionality exists in Eclipse libraries, as well as other places, just to name one case).
diegod
2004-06-29 08:29:03
the problem with open source
I agree that this is a pitfall--but Sun definitely has to find a way to allow for a) proper support of platform elements that have gone into maintenance mode and b) a way to address things that many people need but Sun can't provide. The JCP is a bit too opaque for me, and OSS has some advantages in this regard (if it was properly managed). However, OSS or not OSS, Sun still needs to address the problems of evolution of the platform, one way or another.
diegod
2004-06-29 08:33:48
An Honorable stand
"Java's lack of direction is symptomatic of the industry in general." This is a good point. As I mention in other comments, Sun setting a direction for Java would already be a step forward, and maybe that involves leading without reacting so much (I think that Java has led in a number of areas though). My main argument is that open source might be good, but the bigger problem is direction rather than whether to open source or not--and that maybe open source is considered the "automatic solution" because it has some of the benefits I mention at the end, but I haven't seen many others articulate clearly why OSS would be preferable over the current half-open/JCP process, when the elephant in the room is the nearly constant shift in priorities at the platform level.
diegod
2004-06-29 08:38:41
Open-Source worse than abandonment?
Oops sorry for the runaway italics there. :)
adam@devtty.net
2004-06-29 14:17:20
Who Benefits?
I agree with the main thrust of your argument- Opensourcing and then abandoning something is no good either, to paraphrase.


To me opensourcing java at this point is less about "improving java" then getting java as part of the opensource stack.


C++, C etc. =gcc, glibc
perl, python, php, gtk, qt.


These are languages/libraries that we can build solutions against. Few linux people (unless paid) are choosing Java for that because to most of us, that aren't in the Java community, it appears that Sun could pack up and leave at the drop of a hat.


I don't see building software on top of the Sun Community Source License (which isn't much better than "Shared Source") as good for my software or the software I choose to invest my user attention to.



mrd
2004-06-29 23:29:45
Java, patents, and the C# threat
There are a couple of reasons why Java benefits from being open-sourced as I see it.


These are:


1) Adoption by a new demographic, the open-source community. Right now, Java isn't widely used in the open-source community. Why? Debian GNU/Linux won't ship Sun's java by default because it's not free (as in freedom), simple as that. Neither will Red Hat. And now you've just lost 80% of the open-source developer market. The less popular distributions, including the BSDs, have to jump through hurdles to get Java running. Sun can't support all versions of the top 10 OSS operating systems - they should let the community do that.


And then there's the patent threat. Right now (for example) the GNOME project is agonising over giving Java or C# the tag of preferred language. But that hasn't happened yet because both languages have the cloud of patents hanging over them. If Sun wish for Java to be adopted by the OSS community, they need to at least make a declaration that they won't enforce patents on [L]GPL'd software written by anyone.


And moving on - gaining OSS developers mean Java benefits by havng a new group of people fix Sun's bugs for them. No, not magic pixie dust, but if you can gain the developer maind share, the project benefits, cf OpenOffice, Evolution etc. Large code bases can attract outside developers, and if they can, it means you gain more eyeballs for the code base. With the ever increasing number of APIs that are included in Java, someone needs to be maintaining and improving them - no one company can sustainably do all that, especially if they aren't charging for the software in the first place.


2) Open-Sourcing Java is a proactive step. At a time when I'm seeing C# selected as a langauge of choice over Java, Sun need to do something to create excitement surrounding Java. Even though Java is a vastly more stable platform, and has more software enginneers working on it, there is an ever increasing pressure from customers to move to .Net. Why? Microsoft are experts at building hype. Here at my job we had 60-70 developers working in Java - now that number is down to 5. All new projects are working in .Net because it is winning the marketing war.


Open-sourcing Java also gives the benefit of placing it outside the clenched fist of any one company's control. Sun have attempted it by improving the JSP process, but Microsoft did it better through ECMA and ISO standardisation. Following suit here won't be enough - Sun needs to let Java go so it can prosper. Trying to hold it tight is like squeezing a bar of soap, it will just slip away.



As an aside, I like Java. I want Java to succeed, but what I'm seeing is that C# is overtaking in the marketplace, and then it's only a matter of time until it starts improving technically. Sun need to be proactive, and the best way to do that is to embrace the ever-growing OSS community and make Java open-source.

revdiablo
2004-06-30 11:20:09
the problem with open source
Anyone and his cat could (and in many cases would) just release an incompatible version of the JDK and JRE


Anyone and his cat is free to continue using the official release. This arument raises a big yawn in anyone who has worked with open source for a few years. Forks happen, but they don't cause major problems. Stop worrying so much.


DLL hell ... is back with a vengeance


Again you have overstated the importance of an actual problem. Sure, there are incompatibilities. In general, though, people who write libraries are mindful of the problem and work very hard to avoid it. Try actually using the stuff for a little while before delivering a hyperbolic rant.


crippling compatibility problems between Linux distributions


I'm not sure exactly what you mean here. Config files aren't always in the same place? Binary packages for some distros don't work perfectly on different systems? There's always the source. You (or someone else, if you are afraid of a compiler) can build it for the other platform, and make a new package. Then anyone using the other platform can use your package. If you truly think these are crippling problems, then I'd hate to see what happens when real problems arise.