www.abc.net.au: Downloading streaming TV Shows from ABC iView (rtmp:// .flv)

Downloading streaming TV Shows from ABC iView

by Andy Botting (andybotting.com)

I’ve done a little bit of work since my last post on this, and a couple of people have asked for my stuff. Here goes.

Firstly, you can use rtmpdump to download the iView stream. You’ll need to download rtmpdump 1.4 and compile it yourself. It should just take a ‘make’ as long as you have all the requirements.

When iView starts, it first requests an XML config file, from the URL http://www.abc.net.au/iview/iview_config.xml

<?xml version="1.0" encoding="utf-8"?>
<config>
<param name="authenticate_path"   value="http://202.125.43.119/iview.asmx/isp" />
<param name="media_path"          value="flash/playback/_definst_/" />
<param name="media_path_mp4"      value="flash:mp4/playback/_definst_/" />
<param name="server_streaming"    value="rtmp://cp53909.edgefcs.net/ondemand" />
<param name="server_speedtest"    value="rtmp://cp44823.edgefcs.net/ondemand" />
<param name="xml_help"            value="iview_help.xml" />
<param name="xml_channels"        value="iview_channels.xml" />
<param name="xml_series"          value="http://www.abc.net.au/playback/xml/rmp_series_list.xml" />
<param name="xml_thumbnails"      value="http://www.abc.net.au/playback/xml/thumbnails.xml" />
<param name="xml_classifications" value="http://www.abc.net.au/playback/xml/classifications.xml" />
<param name="xml_feature"         value="http://www.abc.net.au/playback/xml/iview_feature.xml" />
<param name="xml_feature_home"    value="http://www.abc.net.au/playback/xml/iview_homepage.xml" />
<param name="server_time"         value="http://www.abc.net.au/iview/time.htm" />
<param name="thumbs_path"         value="http://www.abc.net.au/playback/thumbs/" />
<param name="base_url"            value="http://www.abc.net.au/iview" />
<param name="channel_id_arts"     value="2260366" />
<param name="channel_id_news"     value="2186765" />
<param name="channel_id_docs"     value="2176127" />
<param name="channel_id_shop"     value="2186639" />
<param name="channel_id_catchup"  value="2172737" />
<param name="channel_id_kazam"    value="2288241" />
<param name="channel_id_faves"    value="2478452" />
<param name="channels_main"       value="catchup,news,docs,arts,shop" />
<param name="channels_kids"       value="kazam,faves" />
</config>

From this file, you can find out which other XML files you need for the channels and program descriptions. Firstly though, you need a special token, which is like an authorisation string. It’s done by doing a HTTP GET on the authenticate_path, which will return something like:

<?xml version="1.0" encoding="utf-8"?>
<iview xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="iview.abc.net.au">
<ip>124.168.17.31</ip>
<isp>iiNet</isp>
<desc>iiNet Limited</desc>
<host>Akamai</host>
<server />
<bwtest />
<token>daEdOckcEbtaqdmdLasbhcBbCbobAbOaxa5-bjOn1r-8-jml_rFAnL&amp;aifp=v001</token>
<text>iView is unmetered for &lt;a href=”http://www.iinet.net.au/” target=”_blank”&gt;iiNet&lt;/a&gt; customers.</text>
<free>yes</free>
<count>5557</count>
<init>false</init>
</iview>

This is doing a lookup of my IP address, to ensure I’m in Australia, and pass me the token. The token has a short lifetime also, only a few minutes. You then need this token to help you build the URL to request the video stream you want.

To find the programs of a particular channel, you need to grab a URL like this: http://www.abc.net.au/playback/xml/output/catchup.xml.

<?xml version="1.0"?>
<rmp-content xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<channel enabled="true" id="2172737">
<name>ABC CatchUp</name>
<description><![CDATA[Recent best of ABC1 & ABC2 TV]]></description>
<intro></intro>
<ident></ident>
<channel-logo>http://www.abc.net.au/playback/img/chl_catchup.png</channel-logo>
<image id=”258433″ order=”1″>
<title><![CDATA[ABC Catchup Background 09]]></title>
<version id=”1071615″>
<title><![CDATA[1230x564jpg]]></title>
<url>http://www.abc.net.au/reslib/200806/r258433_1071615.jpg</url>
</version>
</image>
<image id=”257912″ order=”2″>
<title><![CDATA[ABC Catchup background 06]]></title>
<version id=”1068909″>
<title><![CDATA[1230x564jpg]]></title>
<url>http://www.abc.net.au/reslib/200806/r257912_1068909.jpg</url>
</version>
</image>
<program-title-list>
<program-title id=”352699″ promo=”false” order=”9″>
<title><![CDATA[Catalyst Series 10 Episode 8]]></title>
<short-title></short-title>
<synopsis><![CDATA[Malaria jumps the gap from monkey to man; could bubbles be a solution to the hard hit  mining industry? And see how a horse trainer applies his skill to the training of elephants, with remarkable success.]]></synopsis>
<publish-date>03/04/2009 12:00:00</publish-date>
<expire-date>17/04/2009 00:00:00</expire-date>
<transmission-date>02/04/2009 00:00:00</transmission-date>
<censorship>G</censorship>
<censorship-warning></censorship-warning>
<website>Go to website</website>
<website-url>http://www.abc.net.au/catalyst/</website-url>
<video-download></video-download>
<video-download-url>http://www.abc.net.au/tv/geo/catalyst/vodcast/default.htm</video-download-url>
<shop></shop>
<shop-url></shop-url>
<category>Science and Technology</category>
<cue-points>
</cue-points>
<video-asset id=”1619127″ order=”0″>
<title><![CDATA[1850flv]]></title>
<url>catch_up/catalyst_09_10_08.flv</url>
<unc-path>catalyst_09_10_08.flv</unc-path>
<duration>27.00</duration>
<file-size>135</file-size>
<thumb>abc_catchup.jpg</thumb>
</video-asset>
</program-title>
<program-title id=”….”>
…more programs…
</program-title>
</program-title-list>
</channel>
</rmp-content>

I’ve shortened the output of the ‘Catch Up’ channel here. This is what you’re likely to see when you get the channel XML file. As you can see, this is describing an episode of Catalyst, and I’ve marked the URL in bold.

TOKEN=`curl -q http://202.125.43.119/iview.asmx/isp | grep token | sed 's/<token>//g' | sed 's/\&amp;/\&/g' | sed 's,</token>,,g' | sed 's/ //g'`; ./rtmpdump --rtmp "rtmp://203.206.129.37:1935////flash/playback/_definst_/catch_up/catalyst_09_10_08.flv" --auth "auth=${TOKEN}" -t "rtmp://cp53909.edgefcs.net/ondemand?auth=${TOKEN}" -o test.flv

This horrible command is getting the token, and stripping out all unncessesary characters, and then passing it onto rtmpdump. You might have also noticed in the command above, I have four slashes in the RTMP url. This is to work around some assumptions that rtmpdump makes about the path. I had made a patch, but in rtmpdump 1.4, you can just use 4 slashes to make it work.

Most of this data came from doing Wireshark packet traces while working with the flash-based iView interface. Also important to note that the programs have an expiry date also. If the command above returns a ’stream not found’ message, you’ll probably need a newer episode.

Batch file using rtmpdump 1.4 or 1.5 and wget for downloading TV Shows from ABC iView (by sge)

@ECHO OFF
setlocal ENABLEDELAYEDEXPANSION

::Change to 1.5 if you have problems
SET RTMPVER=1.4
SET USEPROXY=OFF

::Much better than other methods of downloading though.
::Connects to ABC servers to retrieve a authentication token (which is part of the streams URI)
::Select a channel, then retrieves possible episodes from it.
::Follow the prompts and it should download.

CLS
ECHO.
ECHO ABC iView Downloader by sge
ECHO This script simplifies the process of downloading streams off ABC iView.
ECHO.

ECHO Removing any existing temp files... old authentication tokens... old channel list...
ECHO.
:Start
If exist authenticationtoken.xml del authenticationtoken.xml
If exist channel.xml del channel.xml
SET TOKEN=
SET HOST=
SET ISP=
SET FREE=
SET DOWNLOADER=
SET Choice=
SET DLPATH=
SET TEMPFILENAME=
SET FILENAME=
set LINE=
set LINEOFSHOW=
set SHOW=
set ANSWER=

:RetrieveToken
ECHO.
::No more IP geoblocking now even our friends from overseas can download videos/watch ABC iView. Just enter a proxy in Australia. Uses a Akamai token, but it makes no difference here.
::There isn't much point, but someone will use it.... I bet there is a aussie expat somewhere, that need their fix of Gardening Australia and will thank me :P)
if /i {%USEPROXY%}=={ON} (
ECHO BEWARE OF POTENTIAL PROXY
set http_proxy=114.127.246.36:8080
)
ECHO Using wget to retrieve an XML file from ABC's servers...
ECHO.
::wget.exe -O authenticationtoken.xml http://www2b.abc.net.au/iView/Services/iViewHandshaker.asmx/isp
::Fixed IP.
wget.exe -O authenticationtoken.xml http://202.125.43.119/iview.asmx/isp
ECHO.

ECHO Parsing XML... Retrieving Authentication Token...
for /f "tokens=3 delims=<>" %%a in ('type authenticationtoken.xml ^| find "<token>"') do set TOKEN=%%a
for /f "tokens=3 delims=<>" %%a in ('type authenticationtoken.xml ^| find "<host>"') do set HOST=%%a
for /f "tokens=3 delims=<>" %%a in ('type authenticationtoken.xml ^| find "<isp>"') do set ISP=%%a
for /f "tokens=3 delims=<>" %%a in ('type authenticationtoken.xml ^| find "<free>"') do set FREE=%%a
if /i {%HOST%}=={Akamai} (set DOWNLOADER=alt)

ECHO.
ECHO Authentication token recived from ABC: "%TOKEN%"
ECHO You are using iView host: "%host%" through ISP: "%ISP%"
ECHO.
if /i {%FREE%}=={no} (
ECHO iView is NOT unmetered on your connection. Beware.
)

if /i {%DOWNLOADER%}=={alt} goto :AkamaiToken
IF NOT {%DOWNLOADER%}=={alt} goto :RegularToken
:AkamaiToken
ECHO.
ECHO Your host: %host% uses a different style authentication token.
ECHO You need to do another step.
ECHO.
ECHO Attempting to parse Akamai authentication token...
ECHO.
for /f "usebackq tokens=3 delims=<>" %%a in (`type authenticationtoken.xml ^| find "<token>"`) do echo %%a>token.tmp
for /f "usebackq tokens=1,3 delims=&;" %%a in (`more token.tmp`) do set token=%%a^&%%b
del /q token.tmp
ECHO Parsed authentication token: "%token%"
ECHO.
pause
goto :ChannelPick

:RegularToken
ECHO Your host: "%host%" with authentication: "%TOKEN%" uses the regular downloading command.
ECHO.
pause
goto :ChannelPick

:ChannelPick
CLS
ECHO Which channel do you wish to browse:
ECHO.
ECHO A. Catchup (Recent ABC TV)
ECHO.
ECHO B. News
ECHO.
ECHO C. Arts
ECHO.
ECHO D. Shop (Preview ABC shows)
ECHO.
ECHO E. Docs
ECHO.
ECHO F. The Australia Network (English educational materials)
ECHO.
ECHO G. New Stuff (Recent Kids Shows)
ECHO.
ECHO H. Kazam (Action animation for all ages)
ECHO.
ECHO I. Faves (Great Australian drama series)
ECHO.

ECHO Z. Manually enter path
ECHO.
ECHO.

SET /P Choice=Type the letter and press Enter:
:: The syntax in the next line extracts the substring starting at 0 (the beginning) and 1 character long
IF NOT '%Choice%'=='' SET Choice=%Choice:~0,1%
ECHO.
IF /I '%Choice%'=='A' SET URL=http://www.abc.net.au/playback/xml/input/catchup.xml
IF /I '%Choice%'=='B' SET URL=http://www.abc.net.au/playback/xml/input/news.xml
IF /I '%Choice%'=='C' SET URL=http://www.abc.net.au/playback/xml/input/arts.xml
IF /I '%Choice%'=='D' SET URL=http://www.abc.net.au/iview/xml/shop.xml
IF /I '%Choice%'=='E' SET URL=http://www.abc.net.au/playback/xml/input/docs.xml
IF /I '%Choice%'=='F' SET URL=http://www.abc.net.au/playback/xml/input/australianetwork.xml
IF /I '%Choice%'=='G' SET URL=http://www.abc.net.au/playback/xml/input/newstuff.xml
IF /I '%Choice%'=='H' SET URL=http://www.abc.net.au/playback/xml/input/kazam.xml
IF /I '%Choice%'=='I' SET URL=http://www.abc.net.au/playback/xml/input/faves.xml
IF /I '%Choice%'=='Z' GOTO ManualDLPATH
ECHO.
ECHO.

If exist PossibleShowsTEMP del PossibleShowsTEMP
wget.exe -O channel.xml %URL%
ECHO.
ECHO.
ECHO Possible Shows to download are:
ECHO.

::Parsing channel.xml... Echoing shows on screen in format "LINE. SHOW"... Outputting batch friendly show list...
for /f "tokens=3 delims=<>" %%a in ('type channel.xml ^| find "<url>"') do (
set /A LINE=!LINE! + 1
echo !line!. %%a
echo [!LINE!]%%a[/!LINE!] >> PossibleShowsTEMP
)

if /i {%CHOICE%}=={Z} goto :ManualDLPATH
IF NOT {%CHOICE%}=={Z} goto :AutoDLPATH
:ManualDLPATH
ECHO.
ECHO Manually enter download path
ECHO.
ECHO You must get the paths from iView XML (links in the batch file) or from PossibleShowsTEMP
ECHO.
set /p DLPATH=Enter path to download. For Example: catch_up/chaser_09_03_01 (WITHOUT the .flv part):
ECHO.
ECHO You entered: "%DLPATH%"
ECHO.
:ChooseOutput
set /p FILENAME=Enter Filename to Output:
ECHO You typed: "%FILENAME%"
ECHO.
pause
goto :WhichDownloader

:AutoDLPATH
ECHO.
set /p LINEOFSHOW=Which show do you wish to download (Enter number ONLY)
for /f "tokens=2 delims=[]." %%a in ('type PossibleShowsTEMP ^| find "[%LINEOFSHOW%]"') do set DLPATH=%%a
ECHO.
ECHO Episode Found: %DLPATH%
ECHO.
::Gets a filename from the download path
set TEMPFILENAME=%DLPATH%.flv
for /f  "tokens=2 delims=/"  %%I in ('echo %TEMPFILENAME% ^| find "/"') do (
set FILENAME=%%I
)
ECHO Output Filename: %FILENAME%
ECHO.
pause

:WhichDownloader
if /i {%DOWNLOADER%}=={alt} goto :AlternativeDownloadCommand
IF NOT {%DOWNLOADER%}=={alt} goto :RegularDownloadCommand

:AlternativeDownloadCommand
CLS
ECHO You are using the alternative downloader:
ECHO.
ECHO Ok, you will be requesting "%DLPATH%" using the authentication token "%TOKEN%" from ABC iView servers and saving the stream with the filename "%FILENAME%" using the alternative download command with RTMPdump%RTMPver%.
ECHO.
pause
ECHO.
rtmpdump-WIN32-%RTMPVER%.exe -r rtmp://cp53909.edgefcs.net////flash/playback/_definst_/%DLPATH% -t rtmp://cp53909.edgefcs.net/ondemand?auth="%TOKEN%" -o "%FILENAME%"
goto EndMessage

:RegularDownloadCommand
CLS
ECHO You are using the regular downloader:
ECHO.
ECHO Ok, you will be requesting "%DLPATH%" using the authentication token "%TOKEN%"
ECHO from ABC iView servers and saving the stream with the filename "%FILENAME%"
ECHO Using the regular download command with RTMPdump%RTMPver%.
ECHO.
pause
ECHO.
rtmpdump-WIN32-%RTMPVER%.exe -r rtmp://203.18.195.10/ondemand?auth="%TOKEN%" -y %DLPATH% -o "%FILENAME%"
goto :EndMessage

:EndMessage
ECHO If all went well, "%DLPATH%.flv" should have been downloaded as "%FILENAME%"
ECHO It can be played with VLC
ECHO You must force 16:9 aspect or it looks squished
ECHO.
ECHO If the download didn't begin, try another stream.
ECHO.
SET /P ANSWER=Do you wish to download another file? (Y/N)?
echo You chose: %ANSWER%
if /i {%ANSWER%}=={y} (goto :Start)
if /i {%ANSWER%}=={yes} (goto :Start)
goto :CleanUp

:CleanUp
ECHO.
ECHO Removing temp files...
If exist authenticationtoken.xml del authenticationtoken.xml
If exist channel.xml del channel.xml
ECHO.
ECHO Done! Quitting...
pause

http://forums.whirlpool.net.au/forum-replies.cfm?t=1212283

[any ANONYMOUS forum user]

Are there any easier to use programs that can download videos from ABC iView? Are videos at abc.net.au protected? Can they be downloaded legally?

Originally Posted by any ANONYMOUS forum user

Are there any easier to use programs that can download videos from ABC iView? Are videos at abc.net.au protected? Can they be downloaded legally?

Yes

Video not protected (only hard to capture).

Or can use screen capture.

Prob not legal to dl vids - but what is.

abc.net.au doesn't use SWF verification or "secure" RTMPE protocol, so it is very easy to download videos from ABC iView. You may use RTMP Flash stream recorders like Replay Media Catcher or Jaksta.

[bulesky]

You can use Allavsoft to download abc.net.au Iview Video or Radio to MP4, AVI, on Mac/Windows, it’s very easy.

[tcnoithat8]

Trong má»™t cÄ?n ph??ng kh??ch hiện Ä‘á??i th?¬ kh?´ng thá»? c?? má»™t bá»™ ban an dep cao cá??p vá»›i kiá»?u d??ng thá»?i th?°á»?ng v?¬ n?? l?* n??i Ä‘á»? tiá??p kh??ch, l?* bá»™ má?·t của ng?´i nh?* n??n cá?§n Ä‘?°á»?c ch?? trá»?ng. Ná??u gia Ä‘?¬nh bá??n Ä‘ang muốn lá»±a chá»?n má»™t má?«u sofa Ä‘á??p cho kh?´ng gian n?*y th?¬ đừng bá»? lá»? nhá»?ng sá??n phá?©m m?* ch??ng t?´i Ä‘em Ä‘á??n cho bá??n h?´m nay nh?©.



Nhá»?ng bá»™ b?*n ghá?? sofa m??n chá??t liệu sồi má»? tá»± nhi??n vá»›i số l?°á»?ng m??n c?©ng theo nhu cá?§u Ä‘á?§u t?° của chủ nh?*. Vá»›i bá»™ đệm sofa hiện Ä‘á??i, má?«u đệm chá??t cá»? may Ä‘?°á»?c thiá??t ká?? gia c?´ng từ nhá»?ng ng?°á»?i thá»? c?? nhiá»?u kinh nghiệm trong nghá»? n??n Ä‘á»™ bá»?n Ä‘á??p của n?? cá»±c lá»›n.