Writing a web client for MP3 streaming in perl.
What is streaming?
By streaming we refer to the decoding of a file consurrently with its
download, i.e. rendering the contents of the file as they become
available, without having to stand by for the entire file to be
downloaded. Streaming is very useful in applications that involve live
content (such as readio transmissions, or newscasts) or media files
that tend to be very large, that is in applications where the user
cannot afford waiting for the download to be completed before the file
can be decoded. Not all file formats can be streamed. Many common media
fromats/compressions that we use everyday, such as GIF or WAV require
the entire file to be available before the decoding can take place.
Formats such as mpeg-3 or RealAudio on the other hand, can be decoded
as the data comes in. Such formats are called collectively streaming
formats.
Taking advantage of the streaming properties of a file.
If you start downloading an mp3 file and stop the download before it's
finshed, you'll then be able to hear the song that you have downloaded
perfectly ok, up to the point where the transfer was interrupted. Going
a step further, if you start the download and also ask your mp3 player
to play the file which is being written to right now, you'll notice
that the decoder plays the file just fine, and if your network
connection can maintain a sufficient throughput, you could listen to
the entire song as it's being downloaded without a single jitter.
Because network connections rarely are too stable, you will find that
it is good practice to let the download do some progress first and then
start the decoder, so that even if your network throughput loses pace
for a while, there will be some data read ahead for the decoder to
munch as the network catches up.. otherwise you might experience jerks
and pauses. This is a common technique, similar to the lookahead buffer
that car stero cd's use to ensure sound continuity even when you hit a
bump on the road.
Automating the process...
While it was fun to experiment with streaming manually, it would be
even better fun if we made a program to do all of it for us: download
the file from the internet and redirect the incoming data stream to the
decoder. Ideally, we would just provide the url of the remote mp3 file
and soon we would be listening to the song hot off the network. All we
need is to make a program that can open a socket to a remote host, use HTTP to request the file from the
server, and than just feed the incoming data from the socket into the
standard input of the decoder/player to which we will open an input
pipe.
Introducing webamp...
And here's our program, webamp. Have a look at the code, which
is pretty basic, and see for yourself how it works.. I will be
explaining the details shortly.
0 #!/usr/bin/perl
1 use Socket;
2
3 my $handler = 'splay';
4
5 my $url = shift @ARGV;
6 $url=~m/http\:\/\/([^\:^\/]*)(?:\:(\d+))?\/(.*)/;
7 my $host = $1;
8 my $port = $2;
9 $port = 80 unless($port);
10 my $file = '/'.$3;
11
12 my $proto = getprotobyname('tcp');
13 socket(SOCK, PF_INET, SOCK_STREAM, $proto);
14 print "Looking up $host..\n";
15 my $sin = sockaddr_in($port, inet_aton($host));
16 print "Connecting to $host:$port..\n";
17 connect(SOCK, $sin) || die "Connect failed: $!\n";
18
19 my $old_fh = select(SOCK);
20 $|=1;
21 select($old_fh);
22
23 print "Requesting $file..\n";
24 print SOCK "GET $file HTTP/1.0\n";
25 print SOCK "Accept: */*\n";
26 print SOCK "User-Agent: webamp $version\n\n";
27 print "Waiting for reply..\n";
28 my $header = <SOCK>;
29 print "$header\n";
30 exit unless($header=~m/200|OK/);
31
32 while($header = <SOCK>) {
33 chomp;
34 last unless(m/\S/);
35 }
36
37 my $content;
38 open(HANDLER, "|$handler") or die "Cannot pipe input to $handler: $!\n";
39 print "Redirecting HTTP filestream to $handler..\n";
40 while(read(SOCK, $content, 512))
41 {
42 print HANDLER $content;
43 }
44 close SOCK;
How it works.
Line 3
This is just the name of the program that we will use as a decoder. It
is important to use a decoder that is capable of accepting standard
input as we are going to be piping the data stream to it.
Lines 5-10
Here we parse the URL that was passed as an argument from the command
line. The URL consists of three components of interest to us. The
hostname, the port and the filepath relatiev to the server's root. A
simple regular expression (line 6) helps us identify each and store
them in variables $host
, $port
and $file
.
Note that if no port is specified we assume port 80, the default HTTP
server port.
Lines 12-17
Here we create a socket and try to connect to the remote server/port.
If all goes well, we wnd up with a filhandle SOCK
bound
to the socket. We will be using this filhandle to communicate with the
remote program. (the web server in that case) Printing to the filhandle
sends data and reading from the filhandle blocks waiting to receive
until incoming data is available to satisfy the read request.
Lines 19-21
In order for this to work we must turn off the buffering of the stream.
Line 20 turns off buffering of the selected stream. We first select (in
line 19) the socket filhandle, then switch off the bufferring, and then
restore the previously selected filhandle (good housekeeping rules
dictate leaving things as you found them, unless there's good reason
for not doing so) which we had saved on our first select()
.
Lines 23-30
Now that our communication channel is set up, we may send our HTTP
request (lines 24-26) to the remote web server. Then we must wait for
the server to respond. We read the first line from the response which
contains the status code and message. If it's 200 OK, this means that
our request can be satisfied and that headers and content follow...
otherwise we abort. (This is a far too simplistic approach at
interpreting server responses, but our main concern here is to provide
a demonstration rather than a sophisticated web client that can deal
with redirects, caching etc)
Lines 32-35
Here we just loop over the headers ignoring them to get to the actual
content. Again this is probably not an appropriate tactic for a serious
web client, but we just want to provide a working demonstration at the
moment.
Lines 37-44
Finally we ger to the fun part. In line 38 we open a pipe to the inpu
of the handler program, and right after that, we start reading through
the incoming data stream and redirecting all data to the handler which
will decode and play the music.
Further Improvement
There is a lot of room for improvement in this program. First of all
one desirable feature would be to also store the incoming data into a
file as well as passing it along to the handler. So if at the end of
the day we like the song, we have it stored somewhere to play it again
later. Another obsious enhancement would be to change the last part of
the program to buffer the incoming stream and keep a lookahead of a few
Kbytes so that we reduce the chance of jitters in the decoding.
Finally, note that this is a generic streamer application after all..
we can use any program as a handler, so with appropriate handler
programs for various formats we could implement streaming of any other
file type. We could even provide the ability to choose the handler from
the command line, so the same program can be applied to all streaming
uses we wish.
Suggested Reading
Online Documentation/Tutorials