Downloading Channel 4 News with rtmpdump

Mick · #1 11-28-2010, 10:19 AM

Hi All,

I have been trying to download a channel4 news bulletin and capture the stream details with rtmpsrv, but if rtmpsrv is running then the video fails to download and play in the browser. The download seems to start (the embedded player shows black) but the video never starts.

The page is this: http://www.channel4.com/news/catch-up/

When I start rtmpsrv I get this output:

Code:

# rtmpsrv 
RTMP Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
WARNING: Trying different position for client digest!

rtmpdump -r "rtmp://brightcove-82.fcod.llnwd.net:1935/a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -a "a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -f "LNX 10,1,102,64" -W "http://admin.brightcove.com/viewer/us1.24.03.04.2010-11-19132629/federatedVideoUI/BrightcovePlayer.swf" -p "http://www.channel4.com/news/catch-up/" -C B:0 -C S:mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4&1290963600000&93fa72aa482f0cefcf263f1c617234e8 -y "mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -o 1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.flv

Closing connection... done!

RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Duplicate request, skipping.
Caught signal: 13, cleaning up, just a second...
ERROR: WriteN, RTMP send error 32 (42 bytes)
Closing connection... done!

ERROR: RTMP_ReadPacket, failed to read RTMP packet header

Running the rtmpdump code as obtained above with rtmpsrv also fails to download the video:

Code:

~ $ rtmpdump -r "rtmp://brightcove-82.fcod.llnwd.net:1935/a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -a "a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -f "LNX 10,1,102,64" -W "http://admin.brightcove.com/viewer/us1.24.03.04.2010-11-19132629/federatedVideoUI/BrightcovePlayer.swf" -p "http://www.channel4.com/news/catch-up/" -C B:0 -C S:mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4&1290963600000&93fa72aa482f0cefcf263f1c617234e8 -y "mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -o 1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.flv
[1] 12336
[2] 12337
-bash: 1290963600000: command not found
-bash: 93fa72aa482f0cefcf263f1c617234e8: command not found
[2]+  Exit 127                1290963600000
~ $ RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: You haven't specified an output file (-o filename), using stdout
Connecting ...
INFO: Connected...
ERROR: rtmp server sent error
ERROR: rtmp server requested close

BTW, I don't understand the warning about not specifying an output file, as you can see I have specified an output file.

When I close the rtmpsrv and reload the page the video plays normally in the browser.

A tcpflow capture of the connection (when it works without rtmpsrv) shows this:

Code:

(."....>...S...g....H..n.../)ht.....7.. ..O_.H.........a^..E........Kmo..-..(Q..^B+u....1. ..y...G.T.x.
..}-.Quv.R%..z8=..F."...7.Z...vQ.v>=W.....3..r...J.......[h.R.>..Jm6A..-...|]...~.JH.....7..Z...JsF@sX......m.<.$.i.5!-j4.*nD....,.....Mr9..2>.-...W..?.E..B...8@8k.4..(....u...;..F..O...G.#.,.93v.....\..4.Kh..H.U....L%...#.."x.....].p.W.$$.X{..E$..
.....(..{..v..)....Qx...Gb...I.....Y..&..Y{.rpH......q#a8......%dm.....w.SS.......B..!.MH.._.....ji.g..i+._.J.'e....._.7V4w..r.A.%..............W!....Z...<...~.T.W(.>=.)....7.................connect.?..........app..fa500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateI.d=..flashVer...LNX 10,1,102,64..swfUrl..fhttp://admin.brightcove.com/viewer/us1.24.03.04.2010-11-19132629/federatedVideoUI/Brigh.tcovePlayer.swf..tcUrl...rtmp://brightcove-82.fcod.llnwd.net:1935/a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playe.rId=601325122001&playerTag=&affiliateId=..fpad....capabilities.@m........audioCodecs.@.........videoCodecs.@o.......
videoFuncti.on.?.........pageUrl..&http://www.channel4.com/news/catch-up/..objectEncoding................{mp4:media/1184612030/1184612030_69.0046737001_4ON-HEADS-27-Brightcove16x9.mp4&1290963600000&93fa72aa482f0cefcf263f1c617234e8..c..........&%.

At some point during the connection to the brightcove server used by Channel 4, my PC connects with SSLv2 to 95.140.226.95. I am not sure if this is some two stage authentication to the brightcove server using TCP and if rtmpdump/rtmpsrv can deal with this negotiation.

Any ideas what I'm missing?
--
Regards,
Mick

#2 11-29-2010, 05:26 AM

www.channel4.com: Using Rtmpdump for downloading 4OD videos
www.channel4.com: Any way to capture video from the new 4OD service?

#3 11-29-2010, 11:37 PM

Quote:

Originally Posted by Mick

I had already read the threads you suggested before I posted and I am not sure they are directly relevant to the problem I have reported in my thread. You see, they are talking about 4OD which is a different service as far as I can tell to Channel 4 News and it seems they are using different paths/subdomains and perhaps servers.

If however you know what the problem is in my case can you please explain the solution - it is not obvious to me in the threads you offered.

Either way, for now you may want to unlock my thread so that others can read or contribute too.

Sorry that I have missed that. The thread is opened again.

It seems that you use Linux. Have you tried rtmpsuck?

Mick · #4 12-02-2010, 12:14 PM

Yes, I also tried rtmpsuck and it fails to record anything:

Code:

# rtmpsuck
RTMP Proxy Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
WARNING: Trying different position for client digest!
Processing connect
app: a500/d12?videoId=695769735001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=
flashVer: LNX 10,1,102,64
swfUrl: http://admin.brightcove.com/viewer/us1.24.04.00.2010-11-30183707/federatedVideoUI/BrightcovePlayer.swf
tcUrl: rtmp://brightcove-34.fcod.llnwd.net:1935/a500/d12?videoId=695769735001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=
pageUrl: http://www.channel4.com/news/catch-up/
ERROR: RTMP_Connect1, handshake failed.
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
Closing connection... done!

ERROR: Handshake failed
Closing connection... done!

Unlike rtmpsrv, rtmpsuck allows the browser to complete the connection and play the video. For some reason it does not save it on the disk though.

Let me correct this: rtmpsuck creates a file, in the above example it is:

1184612030_695784304001_4ON-NOON-02-Brightcove16x9.flv

but it is empty (i.e. no data is saved in it).

--
Regards,
Mick

nullacht · #5 02-17-2011, 05:59 AM

Quote:

Originally Posted by Mick

The page is this: http://www.channel4.com/news/catch-up/

Running the rtmpdump code as obtained above with rtmpsrv also fails to download the video:

Code:

~ $ rtmpdump -r "rtmp://brightcove-82.fcod.llnwd.net:1935/a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -a "a500/d12?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -f "LNX 10,1,102,64" -W "http://admin.brightcove.com/viewer/us1.24.03.04.2010-11-19132629/federatedVideoUI/BrightcovePlayer.swf" -p "http://www.channel4.com/news/catch-up/" -C B:0 -C S:mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4&1290963600000&93fa72aa482f0cefcf263f1c617234e8 -y "mp4:media/1184612030/1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.mp4?videoId=690038526001&lineUpId=&pubId=1184612030&playerId=601325122001&playerTag=&affiliateId=" -o 1184612030_690046737001_4ON-HEADS-27-Brightcove16x9.flv
[1] 12336
[2] 12337
-bash: 1290963600000: command not found
-bash: 93fa72aa482f0cefcf263f1c617234e8: command not found
[2]+  Exit 127                1290963600000
~ $ RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: You haven't specified an output file (-o filename), using stdout
Connecting ...
INFO: Connected...
ERROR: rtmp server sent error
ERROR: rtmp server requested close

BTW, I don't understand the warning about not specifying an output file, as you can see I have specified an output file.

The issue is being generated by a lack of quotation marks!

Example (Thu 16 Feb Channel 4 News headlines):

RTMP Server v2.3 (rtmpsrv):

Code:

RTMP Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
WARNING: Trying different position for client digest!

rtmpdump -r "rtmp://brightcove-26.fcod.llnwd.net:1935/a500/d14?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -a"a500/d14?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -f "WIN 10,1,102,64" -W "http://admin.brightcove.com/viewer/us1.25.00.02.2011-02-16131048/federatedVideoUI/BrightcovePlayer.swf" -p "http://www.channel4.com/news/catch-up/" -C B:0 -C S:mp4:media/1184612030/1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.mp4&1297947600000&74f8bff08d705e8765f031ebcff523be -y "mp4:media/1184612030/1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.mp4?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -o 1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.flv

...add two quotation marks:

Code:

-C S:"mp4:media/1184612030/1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.mp4&1297947600000&74f8bff08d705e8765f031ebcff523be"

RTMPDump v2.3:

Code:

rtmpdump -r "rtmp://brightcove-26.fcod.llnwd.net:1935/a500/d14?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -a"a500/d14?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -f "WIN 10,1,102,64" -W "http://admin.brightcove.com/viewer/us1.25.00.02.2011-02-16131048/federatedVideoUI/BrightcovePlayer.swf" -p "http://www.channel4.com/news/catch-up/" -C B:0 -C S:"mp4:media/1184612030/1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.mp4&1297947600000&74f8bff08d705e8765f031ebcff523be" -y "mp4:media/1184612030/1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.mp4?videoId=792537078001&lineUpId=&pubId=1184612030&playerId=601325122001&affiliateId=" -o 1184612030_792619822001_4ON-HEADLINES-16-Brightcove16x9.flv

Code:

RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:   duration              77.44
INFO:   moovPosition          28.00
INFO:   width                 720.00
INFO:   height                406.00
INFO:   videocodecid          avc1
INFO:   audiocodecid          mp4a
INFO:   avcprofile            66.00
INFO:   avclevel              30.00
INFO:   aacaot                2.00
INFO:   videoframerate        30.00
INFO:   audiosamplerate       44100.00
INFO:   audiochannels         2.00
INFO: trackinfo:
INFO:   length                2322000.00
INFO:   timescale             30000.00
INFO:   language              eng
INFO: sampledescription:
INFO:   sampletype            avc1
INFO:   length                3415039.00
INFO:   timescale             44100.00
INFO:   language              eng
INFO: sampledescription:
INFO:   sampletype            mp4a
8401.314 kB / 77.42 sec (99.9%)
Download complete

Mick · #6 02-20-2011, 06:31 AM

Yes!

The problem was the missing quotation marks. Why are they being missed out? Is this a browser problem, or an rtmpsrv problem?

Do I need to file a bug report?

Thank you very much for your help!