ONJava.com -- The Independent Source for Enterprise Java
oreilly.comSafari Books Online.Conferences.


AddThis Social Bookmark Button

Java Media Development with QuickTime for Java

by Chris Adamson

Now that Sun's Java Media Framework can't even play MP3s anymore — support was removed in August due to what Sun calls a "licensing issue" — its collection of supported media formats and compression schemes (codecs) has dwindled to near-uselessness. The JMF's powerful plug-in architecture allows developers to expand JMF's capabilities, however, and that's exactly what this article will do, by using the rival media API, Apple's QuickTime for Java.

In this first part, we'll consider how JMF's design leaves the door open to improve its capabilities. In the second part, we'll get into the details of QuickTime for Java to deliver more media support.

Opening up Java Media Framework

To handle a given piece of digital media, an application has to know how to handle the following:

  • Format — how the contents are arranged in a file or network stream.
  • Multiplexing — how multiple media streams are put mixed together into a single byte-stream. While it might be convenient to write all the video data to a file and then all the audio, the resulting file could be difficult or impossible to play at a consistent speed, so you "multiplex," or interleave, the streams together, putting the pieces of each stream that represent the same time close to one another.
  • Encoding — how the media is encoded and (usually) compressed.
  • Rendering — how to present the decoded/decompressed data to the screen, speakers, etc.

Source Code

Download the source code for this article.

Sun's JMF implementation comes with classes that can only handle a handful of the possible formats and codecs you're likely to encounter on the Web. According to the supported media types page, the most popular video formats JMF supports in its all-Java version are the deprecated AVI format and QuickTime's .mov file format ... not Windows Media WMA and WMV, or RealMedia .rm formats. And just because a format is supported doesn't mean a given clip in that format will play. For example, JMF can't handle DivX AVIs or QuickTime files that use the popular Sorenson video codecs.

Extending JMF

The good news is that you're not stuck with this modest media support, thanks to the plug-in architecture. The design of JMF allows it to decide at runtime what code to use to handle a given format, multiplex scheme, and encoding. It does this by carefully parceling out responsibility for format handling, demultiplexing, decoding, and rendering into different classes, then using reflection to discover what kinds of handlers are available.

Extending JMF often means writing new DataSources and/or Players or Processors. A DataSource represents both the media's organization and access to it. In other words, you'd look to a DataSource to determine the media's content type and to open a stream to the media. A Player represents the ability to "play" the media, that is to say, to start reading and decoding the time-based data, and presumably to render it to the screen and/or speakers. Player has a subclass called Processor that allows lower-level access to the decoded media, which you might use to add effects or "transcode" to another format. To keep things simple, in this article, I provide code for only a small subset of Player-defined functionality.

You don't always instantiate DataSources and Players directly (although we did force the issue of our own special .jar-file-handling DataSource in a previous ONJava.com article). Instead, you ask a class called Manager to try to find an appropriate DataSource for a MediaLocator (which is basically a wrapper for a URL), and then an appropriate Player for a DataSource.

How Manager Works

In each case, Manager takes a list of known package prefixes, combines that with a standardized subpackaging and class-naming scheme, and looks for the resulting class in the CLASSPATH. For DataSources, the subpackage path is media.protocol.protocol-type, where protocol-type is the protocol part of the URL — for example, http or file. So, for the default package prefixes javax, com.sun, and com.ibm, the Manager would try to handle an http:-style URL by searching for the classes, respectively:

  • javax.media.protocol.http.DataSource
  • com.sun.media.protocol.http.DataSource
  • com.ibm.media.protocol.http.DataSource

The scheme is similar for Players, except that the subpackage path needs to incorporate the content-type, such as video.mpeg or audio.midi. The basic subpackage path is media.content.content-type.Handler. Yes, Handler, not Player ... it's kind of weird that way. There's also a special rule for Players: if no class is found for a given content-type, the CLASSPATH is searched again with unknown for the content-type. Generally, this "unknown" class is expected to be a "catch-all" player, quite possibly an OS-specific native player. So for video.quicktime content-type and the default package-prefixes as above, Manager would search for six classes:

  • javax.media.content.video.quicktime.Handler
  • com.sun.media.content.video.quicktime.Handler
  • com.ibm.media.content.video.quicktime.Handler
  • javax.media.content.unknown.Handler
  • com.sun.media.content.unknown.Handler
  • com.ibm.media.content.unknown.Handler

Each time it finds one of these classes, Manager attempts to call the Player's setSource() method. The search is over when it finds a Player that doesn't throw an exception when setSource is called.

By the way, in our implementation there's no difference between the DataSource for the http, file, and rtsp protocols, so there's a single com.mac.invalidname.punt.media.protocol.DataSource class, and the protocol-specific subpackages have trivial subclasses of this class.

Using the JMF Registry

To control the behavior of Manager, JMF provides the JMFRegistry application, which allows an end user to inform JMF of new plug-ins and to control some of Manager's behavior.

To use the JMF Registry, use the executable that comes with the OS-specific JMF release, or in the all-Java JMF, enter at the command line:

     java -classpath $JMFHOME/lib/jmf.jar JMFRegistry

Of interest to this article is the "Packages" tab. This defines the list of package prefixes used above, in the order in which they will be tried.

When you're ready to try the JMF-to-QuickTime bridge, run the JMF Registry with the punt.jar file in your CLASSPATH. Use the Add buttons to add com.mac.invalidname.punt to both the Protocol Prefix List and the Content Prefix List, then select the package and use the "Move Up" buttons to make it the first choice in the lists. The result should look like this:

JMF Registry: Packages tab
JMF Registry: Packages tab

Click "Commit" to save your changes to the registry. The Manager will now make com.mac.invalidname.punt the first package it tries when searching for DataSources and Players.

Trying it Out

To get a taste of the JMF-QTJ bridge, make sure you've downloaded and installed the SDKs for JMF (at least version 2.1, latest is 2.1.1c) and QTJ. Since we want to play MPEG-4 files, and since the code uses the new JQTCanvas class, be sure you have QuickTime 6. Windows users should do a custom-install to make sure QuickTime for Java gets installed or updated (it's not installed by default on Windows). After installing, try out the simple demos like JMF's JMStudio and QTJ's PlayMovie, just to make sure you've got any CLASSPATH issues resolved. Note that QuickTime is only available for Windows and Mac — sorry, Linux folks.

On Windows, the Makefile I wrote assumes you're running Cygwin; be sure to export OSTYPE=cygwin to get file and path separators handled correctly. On Mac OS X, there's a curiosity about the QTJava.zip file: it's in your CLASSPATH when running a Java application, but not when compiling with javac or jikes. The Makefile deals with this by always putting QTJava.zip in the CLASSPATH for you.

The code includes a "sanity check" application, launched from the Makefile with make sanitycheck. It shows off the supported methods by setting up an AWT Frame with the movie and its control bar in a non-standard location, playing the media for a few seconds, muting the sound, blasting the sound, stopping the movie, jumping halfway into the media, grabbing the current frame, and then playing backwards. Here's what it looks like:

Screenshot of make sanitycheck with MPEG-4 iMac ad
Screenshot of make sanitycheck with MPEG-4 iMac ad

It takes a URL on the command line — the default in the Makefile is an MPEG-4 clip of one of Apple's iMac advertisements, but the SANITYURL argument is easily changed in the Makefile, overridden by its companion private.mk file, or you can just call com.mac.invalidname.punt.SanityCheck directly.

The code can also be used in JMF's demo media-player, JMStudio. Use make runjmstudio to try it out.

This should handle media formats unsupported by JMF, including MPEG-4, MP3, QuickTime movies with Sorenson Video, and even user-added QuickTime components like On2's open-source VP3 or Apple's optional MPEG-2 component. Curiously, though, while QTJ supports Flash 5, JMF still seems to grab .swf files before I can, and tries to play them with its obsolete Flash 2 player.

Part 2: Closing the Deal with QuickTime for Java

QuickTime Basics

The first thing a QuickTime for Java application has to do is to initialize QuickTime with the QTSession.open() call. Subsequent QTJ method calls will fail if this hasn't been done. It's also important to shut down QuickTime when your app is done. Mac OS X handles this for you when you use the default Quit menu item, and on Windows you typically add a WindowListener on your main window to close down QuickTime before terminating the application. In the case of our bridge, we don't know when the application is actually being shut down, so our static initializer that opens QuickTime also registers a ShutdownHandler to shut down QuickTime when the calling application is going away.

Unsurprisingly, while both JMF and QTJ do some similar things and even have some common method names, their structural approach is rather different. In JMF, a DataSource represents a reference to media (its content type, the ability to start reading data from it, etc.), while a Player represents the ability to actually decode and present that media. In QuickTime, the Movie object represents some of both of these roles. The docs opaquely state, "The movie is not the medium; it is the organizing principle." True enough, but don't get the idea that a Movie is some inert object to be passed around ... in fact, it's the object that has the start() method, and is probably the class you'll get most familiar with.

Because Movie contains functionality represented in JMF by both DataSource (getDuration(), the Positionable interface's setPosition(), etc.) and Player (start(), stop(), setRate(), etc.), it makes sense for our bridge to share a Movie between a DataSource and a Player that we define.

One note before continuing: the included code is an absolute bare-bones implementation of a JMF-to-QTJ bridge, and completely no-ops many methods that exist solely in the JMF world, such as the managing of multiple Controllers and the careful detailing of JMF states. On the latter issue, our Player is always in the Realized state, meaning it's ready to go. Extending the idea to a full-blown JMF implementation, possibly implementing Processor and providing capture ability with the QTJ SequenceGrabber API is, as always, left to the reader as an exercise.

Pages: 1, 2

Next Pagearrow