Java SE Media (Or Not) at JavaOne

by Chris Adamson

So, it's been just over a week since my BoF on Java Media. But let's start with some context about Desktop Java as a whole...


18 Comments

Jeff
2007-05-17 23:52:25
Java and JavaFX and all the Java ecosystem Rocks!. I'm coding right now some examples of JavaFX with the eclipse plugin it's great. Nice article and Great Ideas for Java Media, Keep it up the good work.
kodeninja
2007-05-18 01:17:49
Hey Chris!
A very nice article that highlights a serious limitation with desktop java.


I was wondering, couldn't we use something like
Java --> JNA --> GStreamer to achieve what you're talking about, with minimal amount of fuss.

dave
2007-05-18 04:34:01
Another crazy idea for you.


Background: The HTML5 guys are trying to decide what to do about a "video" tag, analogous to the "img" tag in the next version of HTML. The big stumbling block being codec choice. Firefox, Opera etc. want Ogg Theora + Vorbis (cos it's Free), Apple want H.264 + AAC because they've already paid the licence fees and it's got existing chip support for portable hardware (iPod and iPhone). Microsoft probably want WMV/A but would probably be happy if the web just didn't support this giving Silverlight a chance to beat Flash. If no agreement is reached then HTML5 might end up specifying no codec, or a short list which you only need to support one of, or just some suggested codecs and then see where the market goes.


Elsewhere, the Gstreamer folks already have java players for Free formats (http://www.flumotion.net/cortado/). I'm sure they could be gussied up with some JavaFX to look more like the players you see on YouTube et al.


If newer versions of Firefox and Opera support Ogg Theora natively, and you could use javascript to rewrite the "video" tag into a call to the standard Java applet for older versions of those browsers as well as Safari and IE, then you've got a video delivery platform with a pretty good reach and with a few upgrade paths (upgrade or switch to Firefox/Opera, or upgrade/install Java) for those that don't have it installed by default.


It would be good if Cortado supported JSpeex too (http://jspeex.sourceforge.net/) as an alternative to Flash's NellyMoser codec. I've mentioned mostly video here but obviously the same applies to audio. Sites like Last.fm and Youtube are the only things I use flash for, I'd love to see them redone in web standards or Java.


dave
2007-05-18 04:42:02
And funnily enough, since you mentioned a Java Gstreamer:


"This is Cortado, a multimedia framework for Java written by Fluendo. It contains: JST, a port of the GStreamer 0.10 design to Java"


from https://core.fluendo.com/flumotion/trac/browser/cortado/trunk/README

Andy DePue
2007-05-18 08:48:33
Java needs good multimedia! I just posted this to a Sun blog: Java (and JavaFX) MUST have good video and audio support if it is going to have any relevant place on the desktop or in rich client applications. And the video/audio support must be freely available and usable. The Java platform simply does not have this right now. I don't care if it is a new video format for Java, as long as tools are provided to use it and convert from other formats - AND, as long as the quality is good. OGG is a perfect format candidate, and I would like to see Sun look into some kind of relationship with Fluendo to offer a free 100% Java codec/player. Unless it is cheaper for Sun to just develop the codec and tools themselves.
David Herron
2007-05-18 12:08:39
Great points. One little twitchy detail is there is a third model for recording an online interview... the setup is somewhere on podcast411.com but the site is so badly disorganized I can't find it. The idea is to connect the line output from your computer to a mixer board, and connect your microphone to the same mixer board. Make sure the line output is fully skewed to one stereo channel, and your voice is fully skewed to the other stereo channel. Then you do the mixer output into a digital recorder which could be another computer or a dedicated recorder like the M-Audio handheld thingy. This would work for interviewing a single person, but the multi person conference call such as The Java Posse would not work so well.
Shai Almog
2007-05-19 00:41:27

As you mentioned in your slides, who pays the license fees for GStreamer?

No one!

This is a lawsuit waiting to happen, no one will sue a bunch of GPL developers but this would eliminate the ability to build commercial applications.

Allow me to offer a third hybrid approach that takes the best of both approaches. Build as much as possible in Java and only integrate in "licensing sensitive levels" with the native code. This is very similar to the JMF approach (where codecs were native) with one major difference:

Unlike JMF such an API shouldn't rely on CODEC's being written for it but rather rely on QuickTime codecs where available or DirectShow codecs where available etc... So the portion that needs to be ported is the codec support element.

Wayne Meissner
2007-05-19 04:50:05
If you want java bindings for gstreamer, I've started a project for them over on google: http://code.google.com/p/gstreamer-java/


Strangely enough, as kodeninja suggested, this started as a quick sunday afternoon hack using JNA to get video into a JComponent, and kind of snowballed from there.


I haven't done any release yet, but if you use the latest JNA release, you should be able to checkout from SVN and build.

ingo
2007-05-20 05:14:55
I completely agree with many of your points but I'd like to make one amendment: Keep it simple. YouTube kept the client simple by doing conversion on the server. I think that was very smart. Of course, it doesn't work for everything, but then, it doesn't have to to be successful. As a contrast, JMF is far from simple, but in terms of the API and in terms of the implementation.


Additionally, I think we don't necessarily have to d a framework, we could "just" do a library. The gstreamer guys, to give just one example, went through several framework iterations -- and still make quite deep changes. Christian Schaller once mentioned that this was because initially, the user base was from a limited area and as the user base grew to different applications, unexpected requirements cropped up and had to be integrated.


However, a library with certain media-related functions, such as YUV-conversion, capturing from the various devices and so on would be much simpler to do and I think it would go a long way. At the moment, practically every project re-invents these things, over and over again.

James Ward
2007-05-21 07:18:26
decided to take the fight back to Adobe, before the growing Flash ubiquity can finish off Desktop Java and start selling Flex servers to EE shops


Why does this have to be a battle? I think we can all agree that people should use the best tool for the job. There are places where that tool is Flex and places where it's Java. Turning this into a battle is counter-productive for those of us who use Java AND Flex together. As far as "selling Flex servers to EE shops"... There isn't a Flex server. We do sell a few servlets that enable some additional methods of communication between your Flex front-end and your Java back-end, but those are not always necessary and there are open source alternatives.


While painting Flex as some evil adversary may be good for getting developers excited about fixing the media problems with Java, in reality it's not how those of us who are using Flex AND Java together see the world. We see the two technologies fitting extremely well together.


If you haven't already done so you may want to check out flex.org/java to see how Flex and Java work together, not against each other.


Chris, I really respect you and think you have a lot of good things to say but the attacks on Flex/Flash are unnecessary. Can't we all just get along?


-James (Adobe)

rr
2007-05-21 08:53:28
Porting GStreamer to Java, seems to be an excellent idea!
Chris Adamson
2007-05-21 10:25:46
While painting Flex as some evil adversary may be good for getting developers excited about fixing the media problems with Java, in reality it's not how those of us who are using Flex AND Java together see the world. We see the two technologies fitting extremely well together. [...] Chris, I really respect you and think you have a lot of good things to say but the attacks on Flex/Flash are unnecessary. Can't we all just get along?

James, I don't think I'm in any way portraying you guys as "evil". Actually, as I said in my BoF, Adobe's development and handling of its position in the rich client world has been quite benevolent, and with Flash Player 9 now on Linux, it's clear that Adobe's commitment to cross-platform compatibility is sincere and I think it's very much appreciated. You guys are totally not evil.

But this is a genuine competition, in the most honorable sense of the word, for the rich client developer, and it would be nuts to claim otherwise. I mean seriously, for the last two months you guys have had banner ads on ONJava (ergo, targeted to Java developers) with Bruce Eckel saying "I was Thinking in Java, now I'm Thinking in Flex." You guys are competing for the attention and patronage of developers, and instead of Sun rolling over and dying, I'm glad to see them trying to get their house in order and make consumer-side Java more attractive to developers, i.e., more competitive in the chase for developers.

It does have to be a battle. That's what capitalism is. And don't get me wrong: I like it that way.

Rémi Forax
2007-05-21 16:05:15
fluendo have a (perhaps partial) mapping Java mapping to Gstreamer:
https://core.fluendo.com/flumotion/trac/browser/cortado/trunk/src/com/fluendo/jst


Rémi

Pete
2007-05-23 03:21:58
I do an awful lot of java development and flash development when my services are required. No one I know actually likes flash development. Yes you can do nice things and build stuff quick but it is incredibly buggy. It regularly behaves differently in different browser plugins let alone os's and has some serious flaws as soon as you start doing complex desktop like things. Lets not get started on the long standing list of old fundamental bugs and the broken event mechanism. For this reason I hope JavaFX takes off and the media side gets some loving. It really could be a good and well loved platform.
Gavin
2007-05-23 08:08:21
I would love to see a read/write media "Framework", and it has to to be able to capture. Everthing is moving in the direction of capturing video/audio (in the case of nintendo "writing" and movement) and using it to interact/enrich the user experience.


However it is done. Java needs this.

Chet
2007-05-24 13:24:01
Seems like a good time for us to start talking about what Sun would like to do in the core platform for media. See my Media Frenzy blog for a discussion of the Java Media Components project.
Michael Buckley
2007-05-31 06:54:24
Porting GStreamer to Java doesn't make much sense. Many of the plugins are wrappers/adapters of other chunks of native code (e.g. FFmpeg) and before you know it you are re-writing the entire stack. I dont't see anyone re-writing FFmpgeg in Java any time soon.
Nicolas Desmoulins
2007-06-19 08:09:19
We don't need here to implement all the codecs of ffmpeg. We only need a subset, for example: mp3, Ogg Theora, Ogg Vorbis, maybe h264. There already implementation of ogg vorbis and theora (http://www.flumotion.net/cortado/), and I think for mp3 too.


The problem I fear is performances. I mean, Java can be pretty fast and sometimes rivals C code. But lots of native codecs use assembly optimisations, which can dramatically improve the performances. A pure Java implementation can't compete here. So hum... HD video may but difficult here.