Introduction to Java media programming

Learn the basics of Java-based multimedia with the Java Media and Communication APIs

Java as a versatile programming language has gained widespread acceptance in the two-plus years since its introduction. It is widely espoused as having the usual benefits with which we are all familiar: cross-platform portability, a fairly secure runtime environment, strong networking and connectivity APIs, and so on.

However, no one could credibly argue that the Java platform as we know it (the 1.0 and 1.1 core platforms) is a multimedia power house. In fact, far from offering a robust set of multimedia APIs, Java as a platform still is adding a great many features and gaining acceptance. Media support is only now beginning to roll into the core platform (as part of the upcoming 1.2 release, which will include the Java 2D API and Java Sound engine).

These developments show promise for Java's media capabilities. In future installments of this column we'll be exploring the many aspects of this promise.

History of media support (or lack thereof) in Java

Historically, Java has provided weak support for multimedia. The JDK 1.0 and JDK 1.1 core platform releases provided minimal support for sound in applets and no official support of sound in applications. In addition, Sun provided no support for video manipulation, streaming media capture or playback, 2D or 3D graphics (except through the use of basic drawing primitives), or numerous other advanced multimedia services.

Java PlatformAvailabilityMedia support
1.0 (1995 initial release)All browsers, OSs, IDEs (wide availability)Primitive 2D (AWT); limited audio (applets only); no video, 3D, telephony, or speech
1.1 (1997 release)Some browsers, OSs, IDEs (still has problems one year later)Same as 1.0 release
1.2 (September 1998)Not yet released (no availability except beta from Sun)Adds Java 2D, Java Sound Engine; still no built in video, 3D, telephony, or speech

Media support in the Java platform, at least in any well-organized form, historically has been nil. Sun and its partners have set out to change this -- and to complete an important piece of the Java-as-a-platform puzzle -- by designing and implementing the Java Media and Communication APIs.

The Java Media and Communication APIs

There are a number of APIs that fall within the purview of media and communications. These, and what they enable in the Java platform, include:

  • Java 2D -- 2D graphics and image manipulation
  • Java 3D -- 3D graphics runtime
  • Java Media Framework -- playback of synchronized media
  • Java Sound -- software sound processor and MIDI synthesizer
  • Java Speech -- speech recognition and synthesis
  • Java Telephony -- computer and telephony integration

Sun has also discussed several less well-defined Media APIs, both at JavaOne in March 1998 and in previous public forums:

  • Java Advanced Imaging -- advanced 2D image processing.
  • Java Animation -- 2D Animation.
  • Java Collaboration -- media data sharing.

(Please refer to the Resources at the end of this column for the location of Java Media-related presentation materials from this year's JavaOne conference.)

The Media APIs are available in various states of readiness, from being hinted at in marketing literature, to being discussed only in whitepapers, to being available with full specifications and in some cases implementations.

The states of the various Java Media and Communication APIs as of May 11th, 1998 are:

Java Media APIAPI TypeAPI Specification (version)FAQReference implementation (version)Mailing List
Java 2D Core 1.2Yes (1.0 beta)YesYes (in JDK 1.2 beta)Yes
Java 3D ExtensionYes (1.1 alpha2 early access)YesYes (1.1 alpha2 early access)Yes
Java Media Framework ExtensionYes (1.0.2)YesYes (1.0.1), plus Intel Windows NT/95 implementationYes
Java Sound Engine, Core 1.2; API, ExtensionNoYesEngine, yes (in JDK 1.2 beta releases); API, noYes
Java Speech ExtensionYes (API, 0.7 beta; JSML, 0.5 beta; JSGF, 0.6 beta)NoNo, but IBM has ViaVoice-based implementationYes
Java Telephony ExtensionYes (1.2)YesNo, but shipping third-party implementationsYes
Java Advanced Imaging ExtensionYes (0.9 early access)YesNoYes
Java AnimationUnknownNoNoNoNo
Java CollaborationExtensionNoNoNoNo

Note that the above refers to publicly available information from Sun and does not include licensee-only releases or non-reference implementations. Most of this information came from links available from Sun's Java Media APIs home page. (See Resources for more information.)

We will explore the details of each of these APIs in later articles. Suffice to say that taken as a whole, these APIs comprise a noteworthy attempt at filling out the Java platform's media capabilities. They promise rich support for common formats and rendering options for both file-based and network-based sound, 2D images, 3D models, video, telephony, animation, speech recognition, and collaborative applications. Let's hope these APIs can deliver on this promise.

So, what's wrong with this rosy picture?

Alas, all is not perfect in Java Media platform land. If all of the above APIs were available today in all Java core platform implementations, the Java-based computing landscape would be rich with mature media flora and fauna. Currently, however, the opposite is true: Most Java Media APIs are implemented only partially, and many APIs still lack reference implementations.

This sheer ubiquity of implementation is first and foremost what Java Media currently lacks. If you refer to the table above, you will see some APIs boast implementations while many others do not. Those with implementations are often available on only a few platforms, likely only those officially supported by Sun (Solaris and Windows NT/95).

For example, Apple, long a bastion in the multimedia camp, only recently has demonstrated any strong interest in Java, and to date has not provided implementations of any of the major Java Media APIs. (In fact, Apple's QuickTime for Java is in some ways competing with the Java Media Framework API.)

In addition, organizations with competing APIs (such as the VRML Consortium, Microsoft and Silicon Graphics with its Fahrenheit initiative, and others) are not lining up to provide Java Media implementations.

Above and beyond these problems, if you as a developer implement your applications using any of the non-core platform Media APIs your app is by definition not "100% Pure Java" according to Sun. Because of this, vendors and developers are left in a wait-and-see position as to what to implement and when to implement it. Discussion is underway at Sun as to how to deal with the Media APIs and other extension APIs, but until a workable decision is made, don't count on a 100% Pure label being slapped on any Media-enabled Java software.

Perhaps the biggest Java Media shortcoming lies in Netscape's and Microsoft's lack of direct support for any of the Media APIs in the current generation of Web browsers. Will Java Activator (formally released May 1 under the name Java Plug-in) rise to the occasion to help remedy this situation? Time will tell, but it is not likely that Microsoft will trash its own media APIs and rush to support Java Media, so contention will persist even if Activator gains some developer support.

The good news? Ubiquity can develop very rapidly with computing technologies, as the Internet has demonstrated repeatedly. Should Netscape pick up the Media mantle en masse, for instance, a very large number of users could have access to Java Media-enabled platforms in relatively short order. Alternatively, Sun may go the route of moving more and more Media APIs into the core platform (as the company is doing with Java 2D and Java Sound, beginning with the 1.2 platform), again resulting in Media ubiquity (contract negotiations and lawsuits aside).

Either way, it is probably a good idea to begin learning about Java Media APIs now so that your business can benefit from them when and if ubiquity happens. And for application programmers, who have the option of shipping a Java Runtime Environment with their code, media support might not be such a major issue anyway.

What's next: The proverbial column roadmap

Everyone in the technology world loves to utilize roadmaps, so here's mine for this Media Programming column. I will try to track the various Java Media APIs so that you can learn about them as implementations become available. That way, you can get your hands dirty with the code as the implementations roll out.

Things to look for in upcoming columns:

  • Information on Media APIs in the core platform beginning with 1.2. Start getting your feet wet with Java 2D and the Java Sound engine.
  • A discussion of the Java 3D API, including a compare/contrast with VRML and OpenGL. As promised in my discussion of Java at Siggraph97. (See Resources below for more on this.)
  • An article discussing the Java Media Framework. This will be a revisitation of sorts, discussing what has changed in the API, with continuing examples from my previous JMF article. (See Resources.)
  • Integration of Java Media into and with JavaBean components.
  • Backward compatibility. How to use the Media APIs and yet degrade gracefully for platforms that lack appropriate implementations.
  • Descriptions of Java Telephony, the Java Standard Extensions Architecture, Java Speech, and more.

Conclusions and a blatant request for input

As always, please let me know about topics you want covered. I appreciate any ideas or comments you care to share. I hope this column will provide you with useful information and ideas for the development of media-rich applications.

Bill Day is a software engineer at Silicon Graphics Computer Systems. In addition to writing for JavaWorld, Bill is authoring a book entitled Java Media Players for O'Reilly & Associates. When Bill is not writing or programming, he loves to travel with his wife, speak French, and enjoy life. Java, c'est magnifique!

Learn more about this topic

  • Sun's Java Media home page contains links to the various individual API pages, whitepapers, and more. The definitive source for Media and Communications API information. http://java.sun.com/products/java-media/index.html
  • Sun's Java Media and Communications whitepaper gives a decent overview of the APIs, as of October 1997. http://java.sun.com/marketing/collateral/media.html
  • A significant number of technical sessions and BOFs discussing Java Media were offered at JavaOne 1998. Sun has put the presentation materials from the sessions online. Such materials can serve as a good reference as to where the technology is at and where Sun sees it headed. http://java.sun.com/javaone/javaone98/tracksTOC.html#06
  • I wrote a short news piece reporting on Java Media-related announcements at JavaOne. http://www.javaworld.com/javaworld/javaone98/j1-98-newmedia.html
  • I also wrote an article for the September 1997 issue of JavaWorld that discussed Java Media at the time and its presence at the Siggraph97 conference. http://www.javaworld.com/javaworld/jw-09-1997/jw-09-siggraph.html
  • A previous Java Media Framework article (JavaWorld, April 1997) was based upon a beta version of the API, and is rather dated in places. Nonetheless, it gives you an idea for how one of the major Media APIs might be used. http://www.javaworld.com/javaworld/jw-04-1997/jw-04-jmf.html