Streaming Media With Linux (Part 1)
If you've been trying out the examples from part one, then you should have a good idea about what's already out there on the Internet radio waves. But perhaps you're thinking that you could do a much better show than anything you've heard so far -- if only you knew where to start. Well, let me show you how you can begin your career as the next famous Internet DJ with some help from RealServer and Icecast.
The complete version of RealServer (RealServer Plus) for Linux is commercially available from RealNetworks.com. It's a proprietary closed source Internet broadcasting solution with a fairly hefty price tag. However, bear in mind that RealServer Plus is industrial-strength software and is certainly more than you are likely to need for desktop network broadcasting.
Thankfully, RealNetworks has provided RealServer Basic for evaluation purposes. This basic server is an excellent package for personal broadcasting, but it's not an all-in-one solution for streaming RealAudio. You'll also need the RealProducer content-creation software in order to make your RealAudio files from pre-existing sound files or prepare live netcasts in RealAudio streams. Again, Real has graciously made available a free basic version of RealProducer. We will use the free versions of both RealProducer and RealServer for our examples here.
Download RealProducer Basic (the link to the basic version is at the bottom of that page) into a temporary installations directory and unpack it with
tar xzvf realproducer_8.5_linux.tar.gz. Read the README file, then run
./install to set up RealProducer Basic for your system. Once RealProducer is installed, you can start converting or creating your own content in RealAudio and RealVideo formats. We'll return to RealProducer, but now let's get a server to send our masterpieces out to the networked world.
Retrieve the basic version of the server from the RealServer download site. The software arrives in a binary installation bundle without execution permission. Run this command to make the installation binary executable:
chmod +x rs8-0-linux-libc6.bin
Next run (as root)
./rs8-0-linux-libc6.bin, follow the instructions at the prompt, and within a few moments your RealServer Basic will be fully installed and ready to use.
Now let's create some content to stream from our new RealAudio server. You can use RealProducer to convert existing soundfiles with this simple command:
realproducer -i first.wav -o first.ra
which will create an RA file with default values for a variety of other options. Here is a more complex command sequence (with line breaks added for clarity) along with its output diagnostics:
[dlphilp@localhost outsider]$ realproducer -i 4_LPC_Studies.wav -o /home/dlphilp/4_LPC_Studies.ra \
-t 4 -a 3 -b "Four LPC Studies" -h "David L. Phillips" -c 1998 \
-q "computer treatments of speech" -n "electroacoustic computer-music" \
-g 6 -r 1 -k 1
RealProducer(r) Basic 8.5
Copyright (c) 1995-2000 RealNetworks, Inc. All rights reserved.
------> Beginning Run: Mon Feb 19 18:25:30 2001
Command line: realproducer -i 4_LPC_Studies.wav -o /home/dlphilp/4_LPC_Studies.ra \
-t 4 -a 3 -b "Four LPC Studies" -h "David L. Phillips" -c 1998 \
-q "computer treatments of speech" -n "electroacoustic computer-music" \
-g 6 -r 1 -k 1
Title: Four LPC Studies
Author: David L. Phillips
Keywords: electroacoustic computer-music
Description: computer treatments of speech
Target Audience: DSL/Cable Modem
Audio Format: Stereo Music
Allow Download: Enabled
Server Type: SureStream
------> Ending Run: Mon Feb 19 18:30:16 2001
-tsets "Target Audience," i.e., intended connection rate
-asets audio quality format (set here for "Stereo Music")
-gsets player compatibility to RealPlayer 5.0 or RealPlayer G2
-rtoggles "Allow Recording"
-ktoggles "Allow Download"
The output message reveals the obvious meanings of the string options; run
realproducer --help for a complete list of the available command options.
Note that the encoding took five minutes, achieving a 3:1 reduction in file size for a 6.5 minute high-quality stream. RealProducer's efficiency may be subjectively evaluated by this comparison of input and output file sizes:
[dlphilp@localhost outsider]$ ls -l 4_LPC_Studies.wav
-rwxrwxr-x 1 root root 68692232 Oct 26 15:47 4_LPC_Studies.wav
[dlphilp@localhost outsider]$ ls -l ~/4_LPC_Studies.ra
-rw-r--r-- 1 dlphilp dlphilp 25349828 Feb 19 18:30 /home/dlphilp/4_LPC_Studies.ra
You can also set up RealProducer for live broadcasting in streaming RealAudio. This very simple command line does the trick:
realproducer -l 2:0
-l indicates a live feed and n:n selects the audio capture source. RealProducer handles live input from WAV or AU soundfiles, Quicktime MOV files, and input directed to
/dev/dsp. The example above uses this last method, setting input to the microphone input to your soundcard. 2:1 selects the Line In of your card, and 2:2 chooses your CD player for input.
Finally, let's assemble a list of RA files to be streamed by RealServer. Open your favorite text editor with a file named something like
my_mix.smil. SMIL (Syncronized Multimedia Integration Language) files are composed in a simple mark-up language and are easy to write. Here's a basic example using the loopback adress:
<smil> <body> <audio src="rtsp://127.0.0.1/4_LPC_Studies.ra"/> <audio src="rtsp://127.0.0.1/moanmix.ra"/> <audio src="rtsp://127.0.0.1/odimix.ra"/> <audio src="rtsp://127.0.0.1/rmw.ra"/> <audio src="rtsp://127.0.0.1/tnp.ra"/> </body> </smil>
Note that you can symlink your RA files into the RealServer Content directory; you don't need to copy or move them.
This example simply plays the files in sequence, but SMIL allows much greater flexibility in the layout, timing, and even bandwidth choices for each audio source. Alas, we haven't the time to get further into SMIL, but readers interested in knowing more about SMIL can consult the excellent documentation in RealAdministrator. We need to keep moving, we're almost ready for our first Internet broadcast in RealAudio ...
Become root user (
su root), go to your RealServer installation directory (
/usr/local/realserver on my system), and verify that the installation process created the server configuration file, an XML textfile named
rmserver.cfg. Accept the configuration defaults for now, and start the server with the following command:
Bin/rmserver rmserver.cfg &
A series of messages will scroll by. They are reports from various self-checks the server will make before startup, such as license verification and restrictions, the number of CPUs on the host, and a list of the media format modules loaded.
The start-up messages also report any errors from a misconfigured rmserver.cfg file. Configuration details are available in the RealServer documentation (in
RealAdministrator/Docs) and in the RealNetworks release guide. That information can be most helpful: Thanks to the error report and the documentation, I learned that I needed to add this IP binding before the server could start:
<!-- IP BINDINGS --> <List Name="IPBindings"> <Var Address_01="0.0.0.0"/> </List>
I also needed to set the CPU count with this block:
<!-- S Y S T E M --> <Var ProcessorCount="1"/>
That setting required a command-line flag, as did a bypass for an error with my
/etc/hosts file. At last, I had my working command sequence to launch RealServer:
Bin/rmserver rmserver.cfg --sct --iehp &
Once your server is running, point your browser to the RealServer administration account via the loopback URL:
The admin port number is a random number assigned during installation. See your
rmserver.cfg file for the correct value for your admin port.
From here you simply point and click your way through the server's administrative tools. The GUI is well-designed (Figure 9), help files are a click away, and the experience is totally painless. Alas, we don't have the space here to fully explore RealAdministrator, but I must commend RealNetworks for the obvious effort put into this excellent tool.
Fortunately, you can safely accept most of RealAdministrator's default values and move on towards our goal of producing and transmitting a streaming RealAudio broadcast; however, you should look at and understand the information regarding the server's mount points. Under the Configure topics, click on General Setup, then click on Mount Points (Figure 10).
The base path indicated here is the path to your SMIL files, the point for requests from network clients for the audio stream. You can now verify the server's operation by opening RealPlayer and entering this URL in the File/Open Location dialog box:
This URL assumes the existence of
my_mix.smil at the mount point seen in Figure 10 and tests it via your local host loopback address (127.0.0.1). If the requested file is valid and is located at the configured mount point (
/usr/local/realserver/Content is the default path), RealPlayer will pop up and begin streaming the first audio source in the SMIL list.
Once you have verified the server's correct operation, you're ready for network broadcasting. Open rmserver.cfg and change all references to
localhost.localdomain (or 127.0.0.1) to your actual IP address. Do the same for your SMIL files, then restart your server with the new configuration. For my first network broadcasts, I changed the audio source settings in
my_mix.smil from this address:
to this one:
By the way, run
/sbin/ifconfig to determine your IP address:
ppp0 Link encap:Point-to-Point Protocol
inet addr:18.104.22.168 P-t-P:22.214.171.124 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1
RX packets:2100 errors:0 dropped:0 overruns:0 frame:0
TX packets:2142 errors:0 dropped:0 overruns:0 carrier:0
You will want the value for the
inet addr, not for
P-t-P. Also, note that I would now open my RealAdministrator with this URL:
Obviously, a static IP address is preferable: You enter the address once and leave it. Unfortunately , my address is dynamically assigned, so I need to change it in my server configuration and my SMIL files. I suppose a nice PERL script could take care of all that automatically, but I'll leave that problem for the more resourceful reader.
Regardless of how you update your IP address, you still need a way for people to connect with your server. Easily accomplished by adding this kind of tag to your Web page:
<a href=my_new.ram>My RealAudio Playlist</a>
The RAM (RealAudio Metafile) format is a simple text file containing the address of your SMIL file. Using my IP address du jour again, here's my own
RAM files should be located at the same site as the Web pages containing them. Now clients can connect to your server via Web links or by entering URLs into RealPlayer's File/Open Location dialog box. You can also configure your links to call and stream to a browser's RealPlayer plug-in, but I leave that procedure as another exercise for the diligent student. If you made it this far, then congratulations are in order: You have successfully installed, configured, and deployed RealServer for streaming your media to the Internet.
Icecast is a free open source streaming server licensed under the GPL. It is easy to install, configure, and use, making it possible for anyone with sufficient software and hardware resources to become an instant Internet broadcaster. Icecast 1 supports streaming MP3s and M3U playlists, and version 2 supports streaming Vorbis files. That news should come as a welcome relief to broadcasters concerned about the eventuality of having to pay Fraunhofer for streaming MP3s over the Internet. Vorbis players and plug-ins exist now, with many more on the way, but Icecast 2 is the first streaming Vorbis server. We'll look at both versions of Icecast, starting with version 1.
Download the latest version of Icecast 1 from the link above into your
$HOME directory and unpack it with
tar xzvf icecast-1.3.8.beta2. Move into your new Icecast directory and run
./configure --help to look at your compile-time configuration options. Finally, run
./configure with your selected options, then
make and (as root)
make install. You will also want to download and build the
shout utility available from the Icecast site. Unpack and build it following the directions for Icecast.
The Icecast installation will create a
/usr/local/icecast hierarchy for its default directory path. The conf and etc directories are of particular importance: They are the default locations for your server and streamer configuration files and your M3U playlists. As with RealServer, your configuration files will need such information as your IP address, server port number, connection limits, and so forth. Complete configuration instructions can be found in the
INSTALL files for shout and Icecast.
Let's prepare an M3U playlist for the shout streamer. M3U files are simple text files listing absolute paths to your MP3s. For instance, here's a section from my test playlist for this example:
The playlist is declared in the conf/icecast.conf file. When you have edited your playlist and configuration files, you can start the server by running 'icecast' in an open xterm. You'll see some messages like these:
Icecast Version 1.3.8.beta2 Initializing...
Icecast comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of Icecast under the terms of the
GNU General Public License.
For more information about these matters, see the file named COPYING.
Starting thread engine...
[01/Mar/2001:16:33:48] Icecast Version 1.3.8.beta2 Starting..
[01/Mar/2001:16:33:48] Starting Admin Console Thread...
-> [01/Mar/2001:16:33:48] Starting main connection handler...
-> [01/Mar/2001:16:33:48] Listening on host 127.0.0.1...
-> [01/Mar/2001:16:33:48] Listening on port 8000...
-> [01/Mar/2001:16:33:48] Listening on port 8001...
-> [01/Mar/2001:16:33:48] Using 'localhost.localdomain' as servername...
-> [01/Mar/2001:16:33:48] Server limits: 10 clients, 5 clients per source, 10 sources, 5 admins
-> [01/Mar/2001:16:33:48] WWW Admin interface accessible at http://localhost.localdomain:8000/admin
-> [01/Mar/2001:16:33:48] Starting Calender Thread...
-> -> [01/Mar/2001:16:33:48] Starting UDP handler thread...
-> [01/Mar/2001:16:33:48] [Bandwidth: 0.000000MB/s] [Sources: 0] [Clients: 0] [Admins: 1] [Uptime: 0 seconds]
-> [01/Mar/2001:16:33:48] Starting relay connector thread...
As with our RealServer example, we're testing Icecast on the loopback address before broadcasting to the network. Now we want to send an MP3 stream to the server:
[root@localhost shout-0.8.0]# ./shout localhost -P itsme
Shout version 0.8.0 - www.icecast.org
[7:38] Size: 7337522 Bitrate: 128000 (94070 bytes/dot)
The server acknowledges the connection:
-> [01/Mar/2001:16:38:35] Accepted encoder on mountpoint /default from localhost.localdomain. 1 sources connected
Now fire up XMMS and point it to http://127.0.0.1:8000 to hear the stream. When you're ready for the network, simply change the loopback references in your Icecast and shout configuration files to your actual IP address, just as we did with RealServer, and you're set to rock the net. Of course, you want to make it easy for people to connect to your MP3 stream, so you might want to add a link like this one to a Web page:
<a href=http://connection.m3u>My Streaming MP3 Show</a>
connection.m3u file contains a single line redirecting the request to the actual mount point:
Make sure that you place the
connection.m3u file on the same web site as the page in which the link appears. Web surfers can then simply click on the link to listen to the sounds of your personal MP3 stream.
This introduction touches only the basic points of getting Icecast up and running. The system also handles live broadcasts, and you can reflect your broadcasts to a larger audience using the public server at
yp.icecast.org. There's much more to Icecast, but I'll leave that exploration to the industrious student. Let's move further on and take a look at Icecast 2, the next generation. ...
Icecast 2 is currently available only in a BitKeeper repository, which means you'll have to obtain and install the BitKeeper software. It is available in source code, but I recommend using the easy-to-install pre-compiled binary. Follow the download directions on the BitKeeper page to obtain your user ID and instructions. Log on to the download site, then retrieve the correct binary for your system (glibc 2.0 or glibc 2.1) and move it to a temporary directory (
$HOME/tmp is good). Go into the holding directory, become the root user (
su root) and run
sh x86-glibc21-linux.bin to install the BitKeeper software.
You must also have the OggVorbis libraries correctly installed on your system. If you ran the OggVorbis examples earlier in this article, then you've already installed the necessary libraries. If not, head over to the OggVorbis nightly CVS, download the package, and install the software (as described in this article). Now you're ready to compile and install Icecast.Note: You must use OggVorbis 1.0 beta 4 or higher to build and run the following examples.
Icecast developer Jack Moffit has placed online a set of instructions for downloading and compiling Icecast 2. The process of retrieving files from a BitKeeper repository is simple, but be sure to strictly follow the instructions on that page. Download the sources for Icecast and libshout, then follow Jack's directions for compiling them (
make install should do it).
Before running the server we need to have some content to stream, so let's create a playlist of pieces in OGG format.
Open your favorite text editor and create a new file named "ogglist." This text file lists the locations of the OGG files you wish to play, as in this example:
/home/dlphilp/soundfiles/oggfiles/4_LPC_Studies.ogg /home/dlphilp/soundfiles/oggfiles/moanmix.ogg /home/dlphilp/soundfiles/oggfiles/odimix.ogg
Now go to your Icecast home directory and edit (as root) the
conf/icecast.xml file to reflect your location, server name, connection port, and so forth. While still in the Icecast home top directory, start the server with this command line:
icecast -c conf/icecast.xml
You will see the XML file displayed, along with some debugging messages; then your Icecast2 server is ready to work.
Icecast will eventually use the
ices utility to send Vorbis streams out to the server, but the current BitKeeper libshout package includes a prototype streamer (appropriately named "example") in the examples directory. Open the example.c program in a text editor and edit the following lines to reflect your build configuration:
conn.ip = "127.0.0.1"; conn.password = "itsme"; conn.mount = "/shouter.ogg";
These settings will set up the example binary to stream content to a virtual mount point named shouter.ogg at the loopback address. You can name the mount point anything you like, as long as it ends with
.ogg. The password must match the password in the icecast.xml configuration file.
make to compile the example, then pass your Ogg playlist to the streamer with this command string:
cat ogglist | xargs cat | example 8000
The pipes route the file names from ogglist to xargs, which parses and cats each file to the example streamer at port number 8000. The server window will report on the connection from the streamer:
connection added.... DEBUG: someone signaled us! DEBUG: header read succeeded... What kind of headers do we have? req_type = source protocol = ice version = 1.0 uri = /shouter.ogg DEBUG: source logging in DEBUG: source logged in!
Now bring up XMMS (the CVS version described above) or ogg123 and point it to this address:
If you use ogg123 make sure you indicate an output device:
ogg123 -d oss http://127.0.0.1:8000/shouter.ogg
FreeAmp is another excellent open source streaming audio player. As of release candidate 2.1, Freeamp includes built-in support for streaming Vorbis; unfortunately, I did not have time to test it, but developer Robert Kaye kindly sent a screenshot of FreeAmp rendering a Vorbis stream (Figure 11). With the addition of FreeAmp, Linux users now have their choice of three streaming Vorbis players, and more are certainly on the way. But let's get back to your server ...
The server streams the OGG files from your playlist to the mount point, where the requesting client will receive the stream. Again, the server will report on the client connection:
connection added.... DEBUG: someone signaled us! DEBUG: header read succeeded... What kind of headers do we have? req_type = get protocol = http version = 1.0 uri = /shouter.ogg DEBUG: client coming in... DEBUG: sending ice-bitrate: 0 DEBUG: sending ice-name: no name DEBUG: sending ice-public: 0 Client added Printing client tree: id = 1 Printing pending tree: Done.
Meanwhile, your player should note the connection, pre-buffer some of the stream (you can set the pre-buffering value in XMMS by configuring the Ogg-Vorbis input plug-in), and start playing. Behold the wonder of streaming Vorbis! (Sorry, I couldn't contain my awe... :)
The author thanks Jack Moffitt for his extensive and invaluable assistance with this article, particularly regarding the material on Icecast 1 and 2. Thanks also to my broadband neighbors Jack Paxton and Josh Flechtner and to the members of the Icecast and Vorbis mail-lists for their help with my real-world tests of RealServer and Icecast.
The developers of Icecast 2 are working hard to put the finishing touches on the public distribution package. Much of what I have presented here will likely change (i.e., get easier), but the example presented here delivers the proof of concept. High-quality streaming audio with Vorbis is a reality, and that is great news for anyone involved with network broadcasting.
The pace of development is fast and furious in the world of streaming audio. Access to broadband is becoming more common, content creation is booming, and viable alternatives to patented and licensed streaming formats have begun to appear. Whether on the client side or the server side, Linux is poised to become the platform of choice for listeners and producers of streaming audio. The major streaming formats are already supported by Linux players, utilities, and servers, and even better things are coming our way fast. As we have seen in this article, the future of streaming audio is in good company with Linux: It's just a natural match.
Dave Phillips maintains the Linux Music & Sound Applications Web site and has been a performing musician for more than 30 years.
Return to the Linux DevCenter.
Copyright © 2009 O'Reilly Media, Inc.