Media: JMF + JAI or ffmpeg + ImageMagick?

by Tim O'Brien

It seems like a simple problem, you want to allow people to upload videos, and then you want to process those videos to Flash Video. (Admit it. Flash Video is the video format for the web at the moment.) Talk to the LAMP crowd (or the Rails crowd), and when you ask them the question: "How would you process media files on the web?" You'll find two popular answers:


  1. FFMpeg for video processing

  2. ImageMagick for image processing



But, don't we do everything in Java through the JMF and the JAI? How would one integrate these utilities into Java?

10 Comments

Taylor
2007-04-09 12:57:37
"you should know that the Ruby programmer down the hall is busily calling out to ffmpeg and launching something revenue generating while you are still researching the intricacies of the Java Media Framework (JMF)."


A profound statement with broader implications. (while you are still researching persistence, MVC frameworks, JMF, Hibernate, ...)

S
2007-04-10 03:27:22
Taylor - agreed.


If you are a small agile company churning out web applications for paying customers - and you are using Java, you probably should take a long hard look at your life choices :)


I encourage anybody who is a Java only web app developer to 'step outside the box' and spend a weekend with Ruby / PHP ( even, dare I say, ASP).


I'm not saying Java is bad or the others a better, but the adage 'when all you have is a hammer everything looks like a nail' is quite true of single tech developers.


Tim O'Brien
2007-04-10 06:46:04
@Taylor, @S, i'm not ready to give up on Java as being something that can be very agile. The problem isn't Java in comparison with Rails, etc. I've beat that dead horse too much already. I thin the issue us that Javaists tend think too much about "the platform", this entry is an example. Just call out the Runtime.exec(), leverage the existing tool, dont try to fit the existing tool into your framework. @S, you are right, 'when all you have is a hammer everything looks like a nail'. We need to encourage every Javaist to step down to scripting, not because Java is bad, but because it's a useful perspective. I wonder how many Java programmers end up opting for the most complex option just because they didn't know of the alternative.


But, I'm not saying that the JMF and the JAI are necessarily bad things. i could see instances where it might make sense, if I were a vendor creating something that had to interoperate with the existing JMF subsystem.


I did have a client not that long ago who was trying to do the same thing as Flickr in Java, and it was a painful experience. In addition to creating a custom MVC framework, the idea was to just use JAI everywhere. Where ImageMagick+ruby could have handled the offline processor in ten lines of code, JAI required an API and hundreds of lines of repetitive code.

Fabrizio Giudici
2007-04-10 13:26:35
This topic of JAI producing inconsistent results when resizing an image is one of the urban myths floating around. People try the bicubic interpolation and is not satisfied by results, then give up. The point is that by design JAI operations are "atomic" and the best way to resize an image isn't. In other words, you just have to combine two or three operations together, but in the end you get high quality results. I wouldn't understand how the NASA guys could use JAI if it delivered inconsistent results.


BTW, a complete discussion about image resizing with JAI is here:
http://archives.java.sun.com/cgi-bin/wa?A2=ind0311&L=jai-interest&P=R9579&I=-3


Of course, one could blame JAI for keeping that information not immediately available to programmers, you have to search through the archives of the mailing list. More in general, what I see is the lack of a broad-view community for Java imaging people (not only for JAI, but also for other APIs) where this kind of information could be made more readily available.


For what concerns the verbosity, yes, JAI requires a lot of repetitive code. Unless you write a wrapper that simplifies the API. For instance take a look at mistral.tidalwave.it (work in progress). In this way you can also implement good quality resizing with a single operation ;-) Staying with Java gives a lot of advantages, including being able to work on grids using a number of technologies, such as Jini.

Tim O'Brien
2007-04-10 13:44:53
@Fabrizio, The NASA reference is telling, sure JAI works, but the point here isn't image processing in an organization that has enough resources to launch a reusable space vehicle. I think JAI has it's place, but it isn't necessarily an agile solution.


Thanks for the Mistral reference, the tutorial looks interesting. Mistral also appears to be in line with something that sits atop a standard API and makes it useable for the end-user - in the same way something like Hibernate allows people to benefit from JDBC without touching it.


Fabrizio Giudici
2007-04-10 15:12:42
@Tim We can discuss whether JAI is agile or not, whether such an organization as NASA is required or not to run it... but I understand that the point of the mentioned blog was about its "inconsistent" results. Well, results are consistent if the API is used properly. :-)


For what concerns JAI agility, among its success stories there is a company delivering a product for the desktop: http://java.sun.com/products/java-media/jai/success/lightcrafts.html - so looks like NASA is not required to run it ;-)


Tim O'Brien
2007-04-10 15:23:16
@Fabrizio, I think that's good, no problems with JAI as being a valid selection. Point of the blog post: if you are looking for media utilities and video processing specifically, ImageMagick and ffmpeg should be considered as standalone alternatives especially on the server-side. The target here isn't vendors who need to deliver cross-platform client-side products. In that case, I'd recommend JAI over bundling something like ImageMagick - that's a no brainer.
danb
2007-04-12 16:48:08
I recently started a project called CastCluster (http://castcluster.blogspot.com/) it does some video conversion... I spent a day or so looking at the sad, sad state of media manipulation in Java... then ended up with ffmpeg, Runtime.exec() and regex for figuring out what kind of media I was looking at and what bit rate it was encoded at.... it works pretty well and I'm happy with the decision.
coderslog
2007-05-02 15:40:09
A couple of JMF code snipets to do some basic things


http://www.coderslog.com/JMF


A neat way to get JMF to act like a VoIP server/client


http://www.coderslog.com/RTP_Server_Using_JMF


2007-05-17 01:32:55
Hey Tim.


I don't think Fobs4JMF can do encoding to different formats ... it just supports playback of various media formats at its current state. I think, even if there is a performance problem as you pointed out, using the Runtime.exec() functionality is the way forward at least for the time being.