www.abc.net.au: Downloading streaming TV Shows from ABC iView (rtmp:// .flv)

Downloading streaming TV Shows from ABC iView

by Andy Botting (andybotting.com)

I’ve done a little bit of work since my last post on this, and a couple of people have asked for my stuff. Here goes.

Firstly, you can use rtmpdump to download the iView stream. You’ll need to download rtmpdump 1.4 and compile it yourself. It should just take a ‘make’ as long as you have all the requirements.

When iView starts, it first requests an XML config file, from the URL http://www.abc.net.au/iview/iview_config.xml

<?xml version="1.0" encoding="utf-8"?>
<config>
<param name="authenticate_path"   value="http://202.125.43.119/iview.asmx/isp" />
<param name="media_path"          value="flash/playback/_definst_/" />
<param name="media_path_mp4"      value="flash:mp4/playback/_definst_/" />
<param name="server_streaming"    value="rtmp://cp53909.edgefcs.net/ondemand" />
<param name="server_speedtest"    value="rtmp://cp44823.edgefcs.net/ondemand" />
<param name="xml_help"            value="iview_help.xml" />
<param name="xml_channels"        value="iview_channels.xml" />
<param name="xml_series"          value="http://www.abc.net.au/playback/xml/rmp_series_list.xml" />
<param name="xml_thumbnails"      value="http://www.abc.net.au/playback/xml/thumbnails.xml" />
<param name="xml_classifications" value="http://www.abc.net.au/playback/xml/classifications.xml" />
<param name="xml_feature"         value="http://www.abc.net.au/playback/xml/iview_feature.xml" />
<param name="xml_feature_home"    value="http://www.abc.net.au/playback/xml/iview_homepage.xml" />
<param name="server_time"         value="http://www.abc.net.au/iview/time.htm" />
<param name="thumbs_path"         value="http://www.abc.net.au/playback/thumbs/" />
<param name="base_url"            value="http://www.abc.net.au/iview" />
<param name="channel_id_arts"     value="2260366" />
<param name="channel_id_news"     value="2186765" />
<param name="channel_id_docs"     value="2176127" />
<param name="channel_id_shop"     value="2186639" />
<param name="channel_id_catchup"  value="2172737" />
<param name="channel_id_kazam"    value="2288241" />
<param name="channel_id_faves"    value="2478452" />
<param name="channels_main"       value="catchup,news,docs,arts,shop" />
<param name="channels_kids"       value="kazam,faves" />
</config>

From this file, you can find out which other XML files you need for the channels and program descriptions. Firstly though, you need a special token, which is like an authorisation string. It’s done by doing a HTTP GET on the authenticate_path, which will return something like:

<?xml version="1.0" encoding="utf-8"?>
<iview xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="iview.abc.net.au">
<ip>124.168.17.31</ip>
<isp>iiNet</isp>
<desc>iiNet Limited</desc>
<host>Akamai</host>
<server />
<bwtest />
<token>daEdOckcEbtaqdmdLasbhcBbCbobAbOaxa5-bjOn1r-8-jml_rFAnL&amp;aifp=v001</token>
<text>iView is unmetered for &lt;a href=”http://www.iinet.net.au/” target=”_blank”&gt;iiNet&lt;/a&gt; customers.</text>
<free>yes</free>
<count>5557</count>
<init>false</init>
</iview>

This is doing a lookup of my IP address, to ensure I’m in Australia, and pass me the token. The token has a short lifetime also, only a few minutes. You then need this token to help you build the URL to request the video stream you want.

To find the programs of a particular channel, you need to grab a URL like this: http://www.abc.net.au/playback/xml/output/catchup.xml.

<?xml version="1.0"?>
<rmp-content xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<channel enabled="true" id="2172737">
<name>ABC CatchUp</name>
<description><![CDATA[Recent best of ABC1 & ABC2 TV]]></description>
<intro></intro>
<ident></ident>
<channel-logo>http://www.abc.net.au/playback/img/chl_catchup.png</channel-logo>
<image id=”258433″ order=”1″>
<title><![CDATA[ABC Catchup Background 09]]></title>
<version id=”1071615″>
<title><![CDATA[1230x564jpg]]></title>
<url>http://www.abc.net.au/reslib/200806/r258433_1071615.jpg</url>
</version>
</image>
<image id=”257912″ order=”2″>
<title><![CDATA[ABC Catchup background 06]]></title>
<version id=”1068909″>
<title><![CDATA[1230x564jpg]]></title>
<url>http://www.abc.net.au/reslib/200806/r257912_1068909.jpg</url>
</version>
</image>
<program-title-list>
<program-title id=”352699″ promo=”false” order=”9″>
<title><![CDATA[Catalyst Series 10 Episode 8]]></title>
<short-title></short-title>
<synopsis><![CDATA[Malaria jumps the gap from monkey to man; could bubbles be a solution to the hard hit  mining industry? And see how a horse trainer applies his skill to the training of elephants, with remarkable success.]]></synopsis>
<publish-date>03/04/2009 12:00:00</publish-date>
<expire-date>17/04/2009 00:00:00</expire-date>
<transmission-date>02/04/2009 00:00:00</transmission-date>
<censorship>G</censorship>
<censorship-warning></censorship-warning>
<website>Go to website</website>
<website-url>http://www.abc.net.au/catalyst/</website-url>
<video-download></video-download>
<video-download-url>http://www.abc.net.au/tv/geo/catalyst/vodcast/default.htm</video-download-url>
<shop></shop>
<shop-url></shop-url>
<category>Science and Technology</category>
<cue-points>
</cue-points>
<video-asset id=”1619127″ order=”0″>
<title><![CDATA[1850flv]]></title>
<url>catch_up/catalyst_09_10_08.flv</url>
<unc-path>catalyst_09_10_08.flv</unc-path>
<duration>27.00</duration>
<file-size>135</file-size>
<thumb>abc_catchup.jpg</thumb>
</video-asset>
</program-title>
<program-title id=”….”>
…more programs…
</program-title>
</program-title-list>
</channel>
</rmp-content>

I’ve shortened the output of the ‘Catch Up’ channel here. This is what you’re likely to see when you get the channel XML file. As you can see, this is describing an episode of Catalyst, and I’ve marked the URL in bold.

TOKEN=`curl -q http://202.125.43.119/iview.asmx/isp | grep token | sed 's/<token>//g' | sed 's/\&amp;/\&/g' | sed 's,</token>,,g' | sed 's/ //g'`; ./rtmpdump --rtmp "rtmp://203.206.129.37:1935////flash/playback/_definst_/catch_up/catalyst_09_10_08.flv" --auth "auth=${TOKEN}" -t "rtmp://cp53909.edgefcs.net/ondemand?auth=${TOKEN}" -o test.flv

This horrible command is getting the token, and stripping out all unncessesary characters, and then passing it onto rtmpdump. You might have also noticed in the command above, I have four slashes in the RTMP url. This is to work around some assumptions that rtmpdump makes about the path. I had made a patch, but in rtmpdump 1.4, you can just use 4 slashes to make it work.

Most of this data came from doing Wireshark packet traces while working with the flash-based iView interface. Also important to note that the programs have an expiry date also. If the command above returns a ’stream not found’ message, you’ll probably need a newer episode.