Java Media Development with QuickTime for Javaby Chris Adamson
Now that Sun's Java Media Framework can't even play MP3s anymore — support was removed in August due to what Sun calls a "licensing issue" — its collection of supported media formats and compression schemes (codecs) has dwindled to near-uselessness. The JMF's powerful plug-in architecture allows developers to expand JMF's capabilities, however, and that's exactly what this article will do, by using the rival media API, Apple's QuickTime for Java.
In this first part, we'll consider how JMF's design leaves the door open to improve its capabilities. In the second part, we'll get into the details of QuickTime for Java to deliver more media support.
Opening up Java Media Framework
To handle a given piece of digital media, an application has to know how to handle the following:
- Format — how the contents are arranged in a file or network stream.
- Multiplexing — how multiple media streams are put mixed together into a single byte-stream. While it might be convenient to write all the video data to a file and then all the audio, the resulting file could be difficult or impossible to play at a consistent speed, so you "multiplex," or interleave, the streams together, putting the pieces of each stream that represent the same time close to one another.
- Encoding — how the media is encoded and (usually) compressed.
- Rendering — how to present the decoded/decompressed data to the screen, speakers, etc.
Download the source code for this article.
Sun's JMF implementation comes with classes that can only handle a handful of the possible formats and codecs you're likely to encounter on the Web. According to the supported media types page, the most popular video formats JMF supports in its all-Java version are the deprecated AVI format and QuickTime's
.mov file format ... not Windows Media
WMV, or RealMedia
.rm formats. And just because a format is supported doesn't mean a given clip in that format will play. For example, JMF can't handle DivX AVIs or QuickTime files that use the popular Sorenson video codecs.
The good news is that you're not stuck with this modest media support, thanks to the plug-in architecture. The design of JMF allows it to decide at runtime what code to use to handle a given format, multiplex scheme, and encoding. It does this by carefully parceling out responsibility for format handling, demultiplexing, decoding, and rendering into different classes, then using reflection to discover what kinds of handlers are available.
Extending JMF often means writing new
DataSource represents both the media's organization and access to it. In other words, you'd look to a
DataSource to determine the media's content type and to open a stream to the media. A
Player represents the ability to "play" the media, that is to say, to start reading and decoding the time-based data, and presumably to render it to the screen and/or speakers.
Player has a subclass called
Processor that allows lower-level access to the decoded media, which you might use to add effects or "transcode" to another format. To keep things simple, in this article, I provide code for only a small subset of
You don't always instantiate
Players directly (although we did force the issue of our own special .jar-file-handling
DataSource in a previous ONJava.com article). Instead, you ask a class called
Manager to try to find an appropriate
DataSource for a
MediaLocator (which is basically a wrapper for a URL), and then an appropriate
Player for a
In each case,
Manager takes a list of known package prefixes, combines that with a standardized subpackaging and class-naming scheme, and looks for the resulting class in the
DataSources, the subpackage path is
protocol-type is the protocol part of the URL — for example,
file. So, for the default package prefixes
Manager would try to handle an
http:-style URL by searching for the classes, respectively:
The scheme is similar for
Players, except that the subpackage path needs to incorporate the content-type, such as
audio.midi. The basic subpackage path is
Player ... it's kind of weird that way. There's also a special rule for
Players: if no class is found for a given content-type, the
CLASSPATH is searched again with
unknown for the content-type. Generally, this "unknown" class is expected to be a "catch-all" player, quite possibly an OS-specific native player. So for
video.quicktime content-type and the default package-prefixes as above,
Manager would search for six classes:
Each time it finds one of these classes,
Manager attempts to call the
setSource() method. The search is over when it finds a
Player that doesn't throw an exception when
setSource is called.
By the way, in our implementation there's no difference between the
DataSource for the
rtsp protocols, so there's a single
com.mac.invalidname.punt.media.protocol.DataSource class, and the protocol-specific subpackages have trivial subclasses of this class.
Using the JMF Registry
To control the behavior of
Manager, JMF provides the
JMFRegistry application, which allows an end user to inform JMF of new plug-ins and to control some of
To use the JMF Registry, use the executable that comes with the OS-specific JMF release, or in the all-Java JMF, enter at the command line:
java -classpath $JMFHOME/lib/jmf.jar JMFRegistry
Of interest to this article is the "Packages" tab. This defines the list of package prefixes used above, in the order in which they will be tried.
When you're ready to try the JMF-to-QuickTime bridge, run the JMF Registry with the
punt.jar file in your
CLASSPATH. Use the
Add buttons to add
com.mac.invalidname.punt to both the Protocol Prefix List and the Content Prefix List, then select the package and use the "Move Up" buttons to make it the first choice in the lists. The result should look like this:
JMF Registry: Packages tab
Click "Commit" to save your changes to the registry. The
Manager will now make
com.mac.invalidname.punt the first package it tries when searching for
Trying it Out
To get a taste of the JMF-QTJ bridge, make sure you've downloaded and
installed the SDKs for JMF
(at least version 2.1, latest is 2.1.1c) and QTJ. Since we want
to play MPEG-4 files, and since the code uses the new
class, be sure you have QuickTime 6. Windows users should do a custom-install
to make sure QuickTime for Java gets installed or updated (it's not installed
by default on Windows). After installing, try out the simple demos like JMF's
JMStudio and QTJ's
PlayMovie, just to make sure
you've got any
CLASSPATH issues resolved. Note that QuickTime is
only available for Windows and Mac — sorry, Linux folks.
On Windows, the Makefile I wrote assumes you're running Cygwin; be sure to
export OSTYPE=cygwin to get file and path separators handled correctly. On Mac OS X, there's a curiosity about the
QTJava.zip file: it's in your
CLASSPATH when running a Java application, but not when compiling with
jikes. The Makefile deals with this by always putting
QTJava.zip in the
CLASSPATH for you.
The code includes a "sanity check" application, launched from the Makefile with
make sanitycheck. It shows off the supported methods by setting up an AWT Frame with the movie and its control bar in a non-standard location, playing the media for a few seconds, muting the sound, blasting the sound, stopping the movie, jumping halfway into the media, grabbing the current frame, and then playing backwards. Here's what it looks like:
Screenshot of make sanitycheck with MPEG-4 iMac ad
It takes a URL on the command line — the default in the Makefile is an MPEG-4 clip of one of Apple's iMac advertisements, but the
SANITYURL argument is easily changed in the Makefile, overridden by its companion
private.mk file, or you can just call
The code can also be used in JMF's demo media-player, JMStudio. Use
make runjmstudio to try it out.
This should handle media formats unsupported by JMF, including MPEG-4, MP3, QuickTime movies with Sorenson Video, and even user-added QuickTime components like On2's open-source VP3 or Apple's optional MPEG-2 component. Curiously, though, while QTJ supports Flash 5, JMF still seems to grab
.swf files before I can, and tries to play them with its obsolete Flash 2 player.
Part 2: Closing the Deal with QuickTime for Java
The first thing a QuickTime for Java application has to do is to initialize QuickTime with the
QTSession.open() call. Subsequent QTJ method calls will fail if this hasn't been done. It's also important to shut down QuickTime when your app is done. Mac OS X handles this for you when you use the default Quit menu item, and on Windows you typically add a
WindowListener on your main window to close down QuickTime before terminating the application. In the case of our bridge, we don't know when the application is actually being shut down, so our static initializer that opens QuickTime also registers a
ShutdownHandler to shut down QuickTime when the calling application is going away.
Unsurprisingly, while both JMF and QTJ do some similar things and even have some common method names, their structural approach is rather different. In JMF, a
DataSource represents a reference to media (its content type, the ability to start reading data from it, etc.), while a
Player represents the ability to actually decode and present that media. In QuickTime, the
Movie object represents some of both of these roles. The docs opaquely state, "The movie is not the medium; it is the organizing principle." True enough, but don't get the idea that a
Movie is some inert object to be passed around ... in fact, it's the object that has the
start() method, and is probably the class you'll get most familiar with.
Movie contains functionality represented in JMF by both
setPosition(), etc.) and
setRate(), etc.), it makes sense for our bridge to share a
Movie between a
DataSource and a
Player that we define.
One note before continuing: the included code is an absolute bare-bones implementation of a JMF-to-QTJ bridge, and completely no-ops many methods that exist solely in the JMF world, such as the managing of multiple
Controllers and the careful detailing of JMF states. On the latter issue, our
Player is always in the
Realized state, meaning it's ready to go. Extending the idea to a full-blown JMF implementation, possibly implementing
Processor and providing capture ability with the QTJ
SequenceGrabber API is, as always, left to the reader as an exercise.
Pages: 1, 2