Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)
-   rtmpdump (http://stream-recorder.com/forum/forumdisplay.php?f=54)
-   -  

Cannot capture ITV streams

(http://stream-recorder.com/forum/showthread.php?t=15984)

Mick 03-29-2013 09:38 AM

Cannot capture ITV streams


 
I just tried to download this ITV video and it gets stuck at the ITV logo. It downloads a short clip with the logo animation and then keeps repeating and repeating the attempt to download it, with errors like this:
Code:

# iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
# rtmpsrv
RTMP Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935

rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand?auth=eaEaVdfcqcDanbEb7bvcTapaOcGdKdya9cvbQaUckcpb3aycNbjbnced_cpbZaQb5aj-brvBf7-eS-bAr3ICwjyswq&aifp=v002&slist=production/" -a "ondemand?auth=eaEaVdfcqcDanbEb7bvcTapaOcGdKdya9cvbQaUckcpb3aycNbjbnced_cpbZaQb5aj-brvBf7-eS-bAr3ICwjyswq&aifp=v002&slist=production/" -f "LNX 11,2,202,275" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf?v=12.18.4" -p "https://www.itv.com" -y "mp4:production/priority/stings/12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.mp4" -o 12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.flv

Closing connection... done!

RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
ERROR: RTMP_HashSWF: connection lost while downloading swfurl https://www.itv.com/mediaplayer/ITVMediaPlayer.swf?v=12.18.4
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 8
INFO: Connected...

rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand?auth=eaEaVdfcqcDanbEb7bvcTapaOcGdKdya9cvbQaUckcpb3aycNbjbnced_cpbZaQb5aj-brvBf7-eS-bAr3ICwjyswq&aifp=v002&slist=production/" -a "ondemand?auth=eaEaVdfcqcDanbEb7bvcTapaOcGdKdya9cvbQaUckcpb3aycNbjbnced_cpbZaQb5aj-brvBf7-eS-bAr3ICwjyswq&aifp=v002&slist=production/" -f "LNX 11,2,202,275" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf?v=12.18.4" -p "https://www.itv.com" -y "mp4:production/priority/stings/12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.mp4" -o 12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.flv

Closing connection... done!

Is this because the URL is on https?

Is there a workaround to this problem?

Can someone please have a go and let me know if they can download it?

chap 03-29-2013 11:30 AM

Re: Cannot capture ITV streams


 
Code:

>rtmpdump -r "rtmpe://cp72511.edgefcs.net/ondemand?auth=eaEdEcAbFagb9dubRc9bxckc
La_dlcYazcrb_brdBcMaUdMcOaed.a4dkcObEbtbsdf-brvC8p-eS-4zs2DAtkwuwq&aifp=v002&sli
st=production/" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf" -y "mp4:
production/priority/rtmpecatchup/1200/32/2-1926-0001-003/2-1926-0001-003_broadch
urch_tx040313_itv1200_16x9-reg.mp4" -o "4.flv"
RTMPDump v2.4 GIT-2012-12-09 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:  duration              2783.24
INFO:  moovPosition          32.00
INFO:  audiocodecid          mp4a
INFO:  width                704.00
INFO:  height                396.00
INFO:  videocodecid          avc1
INFO:  avcprofile            77.00
INFO:  avclevel              31.00
INFO:  aacaot                2.00
INFO:  audiosamplerate      48000.00
INFO:  audiochannels        2.00
INFO:  videoframerate        25.00
INFO: trackinfo:
INFO:  length                133595136.00
INFO:  timescale            48000.00
INFO:  language              eng
INFO: sampledescription:
INFO:  sampletype            mp4a
INFO:  length                69581000.00
INFO:  timescale            25000.00
INFO:  language              eng
INFO: sampledescription:
INFO:  sampletype            avc1
9271.568 kB / 63.36 sec (2.2%)


Mick 03-29-2013 11:39 AM

Re: Cannot capture ITV streams


 
Thanks!

It seems that my iptables redirect command no longer works. :(

I recall that some versions ago the iptables rules changed and it no longer accepts the redirect rule:
Code:

# iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
# /sbin/iptables -L -v -n | grep 1935
#

It doesn't complain either to let me know that something is amiss. Any idea what syntax I should use?

chap 03-29-2013 12:03 PM

Re: Cannot capture ITV streams


 
try use windows:)

karlo2105 03-29-2013 12:49 PM

Re: Cannot capture ITV streams


 
Quote:

Originally Posted by Mick (Post 61121)
Thanks!

It seems that my iptables redirect command no longer works. :(

I recall that some versions ago the iptables rules changed and it no longer accepts the redirect rule:
Code:

# iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
# /sbin/iptables -L -v -n | grep 1935
#

It doesn't complain either to let me know that something is amiss. Any idea what syntax I should use?

Take a look here.:D

Mick 03-29-2013 01:18 PM

Re: Cannot capture ITV streams


 
Thank you karlo2105,

The problem is with my firewall. The iptables syntax suggested in the link you provided no longer works on my setup. I'm reading up now to find out how to change the syntax to make it work.

Otherwise, if I were to use the /etc/hosts file to add the rtmpe address, instead of a iptables rule to achieve redirection, where would I be able to retrieve this rtmpe address from?

Mick 03-30-2013 07:23 AM

Re: Cannot capture ITV streams


 
Quote:

Originally Posted by Mick (Post 61124)
The problem is with my firewall.

No it isn't! I looked into it further. The PC firewall redirection rule is created and applied. Doing some testing I can confirm that rtmpsrv works - but only on some servers! It seems that ITV and Channel 4 are recently having a problem with my set up - or should I say that my setup is no longer working with these servers.

I could really do with some help to troubleshoot this please guys. :confused:

Here's a summary to date. I can capture with rtmpsrv and download with rtmpdump videos like this:

http://www.channel5.com/shows/extrao...episode-13-150

but cannot either capture all the headers or download videos from ITV or Channel 4, like this: https://www.itv.com/itvplayer/broadc...es-1/episode-1

When I start playing videos (recently) from ITV or Channel 4, rtmpsrv gets stuck at the advert server playpath. The main video headers never show up. All I get is:
Code:

# rtmpsrv
RTMP Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935

rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand?auth=eaEddcRbxb9bYdBcQdiada9dDdZdObuaMdmavdHbqdOa.bCcidJdXa3b3d3aLbOajdE-brvUhY-eS-7zp2FBokvuDr&aifp=v002&slist=production/" -a "ondemand?auth=eaEddcRbxb9bYdBcQdiada9dDdZdObuaMdmavdHbqdOa.bCcidJdXa3b3d3aLbOajdE-brvUhY-eS-7zp2FBokvuDr&aifp=v002&slist=production/" -f "LNX 11,2,202,275" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf?v=12.18.4" -p "https://www.itv.com/itvplayer/broadchurch/series-1/episode-1" -y "mp4:production/priority/stings/12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.mp4" -o 12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.flv

Closing connection... done!

RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 8
INFO: Connected...
Duplicate request, skipping.
Closing connection... done!

ERROR: RTMP_ReadPacket, failed to read RTMP packet header

I even tried the string kindly provided by chap yesterday, but it is rejected by the server:
Code:

$ rtmpdump -r "rtmpe://cp72511.edgefcs.net/ondemand?auth=eaEdEcAbFagb9dubRc9bxckcLa_dlcYazcrb_brdBcMaUdMcOaed.a4dkcObEbtbsdf-brvC8p-eS-4zs2DAtkwuwq&aifp=v002&slist=production/" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf" -y "mp4:production/priority/rtmpecatchup/1200/32/2-1926-0001-003/2-1926-0001-003_broadchurch_tx040313_itv1200_16x9-reg.mp4" -o "4.flv"
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
INFO: Connected...
ERROR: rtmp server sent error
ERROR: rtmp server requested close

I even installed rtmpdump from git but it made no difference. :(

Short of going now to buy a MSWindows OS just to run an application that was written for Linux (thanks chap ;) ) is there something in what I posted that can point to a problem and ideally a solution for the errors I've been getting? After all if it works with older streams from channel 5, but not recent streams from ITV or Channel 4 it has to be something to do with the harndshakes and headers that these servers exchange - yes?

In case this is related: I've also tried rtmpsuck but can't get it to work with any server. It fails to capture the playpath even on old channel 5 videos that rtmpsrv/dump work:
Code:

WARNING: HandShake: Type mismatch: client sent 6, server answered 8
Processing connect
app: a800/o10?videoId=919379405001&lineUpId=&pubId=19582164001&playerId=1707001744001&affiliateId=
flashVer: LNX 11,2,202,275
swfUrl: http://admin.brightcove.com/viewer/us20130328.1037/connection/ExternalConnection_2.swf
tcUrl: rtmpe://bt.fcod.llnwd.net:1935/a800/o10?videoId=919379405001&lineUpId=&pubId=19582164001&playerId=1707001744001&affiliateId=
pageUrl: http://www.channel5.com/shows/extraordinary-dogs/episodes/episode-13-150
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
Closing connection... done!

Grateful for any pointers.

karlo2105 03-30-2013 07:39 AM

Re: Cannot capture ITV streams


 
Try with rtmpsuck.

Mick 03-30-2013 08:07 AM

Re: Cannot capture ITV streams


 
You probably missed it. I can't get rtmpsuck to work at all.
Quote:

Originally Posted by Mick (Post 61138)
In case this is related: I've also tried rtmpsuck but can't get it to work with any server. It fails to capture the playpath even on old channel 5 videos that rtmpsrv/dump work:
Code:

WARNING: HandShake: Type mismatch: client sent 6, server answered 8
Processing connect
app: a800/o10?videoId=919379405001&lineUpId=&pubId=19582164001&playerId=1707001744001&affiliateId=
flashVer: LNX 11,2,202,275
swfUrl: http://admin.brightcove.com/viewer/us20130328.1037/connection/ExternalConnection_2.swf
tcUrl: rtmpe://bt.fcod.llnwd.net:1935/a800/o10?videoId=919379405001&lineUpId=&pubId=19582164001&playerId=1707001744001&affiliateId=
pageUrl: http://www.channel5.com/shows/extraordinary-dogs/episodes/episode-13-150
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
Closing connection... done!



chap 03-30-2013 11:50 PM

Re: Cannot capture ITV streams


 
try use latest version rtmpdump
Code:

http://ffmpeg.zeranoe.com/builds/source/external_libraries/rtmpdump-20121230-git-df6c518.tar.xz
try
episode-13-150
example
Code:

rtmpdump -r "rtmpe://bt.fcod.llnwd.net:1935/a800/o10" -a "a800/o10" -y "mp4:videos/19582164001/19582164001_919573411001_b17-C5153270013A4-H264-1500-16x9.mp4?e=1364721723&h=dd695d3f03f9de308c2e53ef7dbdb531&__nn__=659726446001" -o "4.flv"

ITV
need to change the link manually(new auth=bla,bla,bla)-otherwise download not work(Whatever you do not use)

Mick 03-31-2013 04:30 AM

Re: Cannot capture ITV streams


 
Thank you chap!

Quote:

Originally Posted by chap (Post 61158)
try use latest version rtmpdump
Code:

http://ffmpeg.zeranoe.com/builds/source/external_libraries/rtmpdump-20121230-git-df6c518.tar.xz

I have just built the latest git rtmpdump from here: git://git.ffmpeg.org/rtmpdump and it includes all latest commits from 30 March 2013.

Quote:

Originally Posted by chap (Post 61158)
try
episode-13-150
example

Probably I did not explain clearly: I have no problem playing older URLs from www.channel5.com. The rtmp requests are captured fully by using rtmpsrv. Then I can download them using trmpdump. No problem with these older video URLs.

HOWEVER, if I try to download more recent videos from www.itv.com or from http://www.channel4.com/programmes/4od then rtmpsrv is not capturing the rtmp requests completely. I think that this happens because the flash player interface is asking the user to login with username and password (on channel4) or is asking the user to add their post code and accept adult content (on itv). Could it be that this authentication process breaks the rtmpsrv from capturing the complete request?

Quote:

Originally Posted by chap (Post 61158)
ITV
need to change the link manually(new auth=bla,bla,bla)-otherwise download not work(Whatever you do not use)

Can you please explain how am I supposed to change the link? Which part of the link and with what information? For example, when I go to this URL: https://www.itv.com/itvplayer/broadc...es-1/episode-1 the flash player asks for my post code (to check that I am trying to watch the video from the UK) and then it asks me to confirm that I am an adult (because this is a crime thriller not suitable for children). After I enter all this information in the browser I can play the video - but here is where the problems start. The video starts with the itv logo and then playing adverts. It does not continue to the main video after the adverts when I am running rtmpsrv. Meanwhile, rtmpsrv ONLY captures the request to the stream server that shows the itv logo. This is what I get with the latest rtmpsrv from git:
Code:

# rtmpsrv
RTMP Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935

rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand?auth=eaEd0b7aEbycJdwclbRcPdfdbb.cwd5bndJcbb5adcSaod_bWaNdsbLbAatccbWdNaz-brv_Wp-eS-6yv1CBrlAuyr&aifp=v002&slist=production/" -a "ondemand?auth=eaEd0b7aEbycJdwclbRcPdfdbb.cwd5bndJcbb5adcSaod_bWaNdsbLbAatccbWdNaz-brv_Wp-eS-6yv1CBrlAuyr&aifp=v002&slist=production/" -f "LNX 11,2,202,275" -W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf?v=12.18.4" -p "https://www.itv.com/itvplayer/broadchurch/series-1/episode-1" -y "mp4:production/priority/stings/12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.mp4" -o 12334234_Adrobot_rebrand_7secplayersting169_7_495_600.BASELINE.flv

Closing connection... done!

RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:  duration              7.04
INFO:  moovPosition          40.00
INFO:  width                512.00
INFO:  height                288.00
INFO:  videocodecid          avc1
INFO:  audiocodecid          mp4a
INFO:  avcprofile            66.00
INFO:  avclevel              30.00
INFO:  aacaot                2.00
INFO:  videoframerate        25.00
INFO:  audiosamplerate      48000.00
INFO:  audiochannels        2.00
INFO: tags:
INFO:  ©too                Lavf53.2.0
INFO: trackinfo:
INFO:  length                175.00
INFO:  timescale            25.00
INFO:  language              und
INFO: sampledescription:
INFO:  sampletype            avc1
INFO:  length                337920.00
INFO:  timescale            48000.00
INFO:  language              und
INFO: sampledescription:
INFO:  sampletype            mp4a
526.284 kB / 7.02 sec (99.7%)
Download may be incomplete (downloaded about 99.70%), try resuming

If I download with rtmpdump and play 12334234_Adrobot_rebrand_7secplayersting169_7_495_ 600.BASELINE.flv all I get is the video of the itv logo that lasts just a few seconds, not the video which is supposed to follow. The playpath for the main video is different and this is NOT captured by rtmpsrv.

Now, having said all this, the good news is that with the latest build from git I just tried and can record the main video with rtmpsuck! YES! Success at last. :D Previous versions of rtmpsuck would not work at all - nothing saved on the disk. This version works fine. :)

So it seems to me that the rtmp devs are playing catch up with whatever obstacles are being set up by the media providers and the very latest version of rtmp is more likely to succeed.

Thank you all for your suggestions and a BIG thank you to rtmp developers!

Mick 03-31-2013 11:05 AM

Re: Cannot capture ITV streams


 
Just to add that rtmpsuck does not continue to record to the end of the video. At some point it drops the connection, reconnects and then overwrites the previously recorded file! So the first half of the video is erased. :-(

Here are my observations to date with this version of rtmpdump:

Using Chromium browser incognito mode, all adverts in the video are played normally. Every time the video arrives at at an advert break it drops the connection and restarts after the adverts, but does not rename the previous file. As a result the video is overwritten and the recording lost.
:-(

I have not tried all different browsers with and without Adblock Plus, to see if it makes a difference. However, trying Chromium in normal mode with AdBlock Plus, it does not play any adverts and initially over the first 2-3 advert breaks it records continuously. Great! I thought, but unfortunately this does not continue to the end. After say 200-300MB or even less, it drops the connection, reconnects and continues to record, overwriting the file.

This is a bit of a pain because I have to keep an eye on the download to see if it stops, quickly copy the file before it is overwritten and then stitch up all the different files into one recording.

Interested to hear if other browsers (with what settings) do not suffer from this problem.

riddler 04-08-2013 08:39 AM

Re: Cannot capture ITV streams


 
Hi all, I need help with ITV Player too. Posted on the main RTMP thread and didn't get any response. I'm on Windows, used RTMP Helper and RTMP Explore X and can't detect the stream at all. I use a free UK VPN.

I can only download using RTMPDump after someone detects the URL and gives me the download string.

https://www.itv.com/itvplayer/daybreak/08-04-2013

And http://www.itv.com/daybreak/showbiz/keeley-hawes/ which I've been trying to get for the past 2 months.

evol 04-08-2013 09:12 AM

Re: Cannot capture ITV streams


 
The trick is not to catch the ads at the begging of the stream :)

chap 04-08-2013 03:45 PM

Re: Cannot capture ITV streams


 
Quote:

Originally Posted by riddler (Post 61438)
https://www.itv.com/itvplayer/daybreak/08-04-2013

which I've been trying to get for the past 2 months.

Code:


>rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand" -a "ondemand?auth=eaEdZ
cjatdjbAc6cwdwc.cxdEard7bCbadzdQdCaZcfcUb1cPa2d.aqbrd0d_cjb7cB-bryZKl-eS-5zv1LDw
kBtEr&aifp=v002&slist=production/" -f "WIN 11,6,602,180" -W "https://www.itv.com
/mediaplayer/ITVMediaPlayer.swf" -p "https://www.itv.com/itvplayer/daybreak/08-0
4-2013" -y "mp4:production/priority/rtmpecatchup/1200/35/1-9358-0849-001/1-9358-
0849-001_daybreak-(2013)_tx080413_itv1200_16x9-reg.mp4" -o "4.flv"
RTMPDump v2.4 GIT-2012-12-30 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:  duration                7535.83
INFO:  moovPosition            32.00
INFO:  audiocodecid            mp4a
INFO:  width                  704.00
INFO:  height                  396.00
INFO:  videocodecid            avc1
INFO:  avcprofile              77.00
INFO:  avclevel                31.00
INFO:  aacaot                  2.00
INFO:  audiosamplerate        48000.00
INFO:  audiochannels          2.00
INFO:  videoframerate          25.00
INFO: trackinfo:
INFO:  length                  361719808.00
INFO:  timescale              48000.00
INFO:  language                eng
INFO: sampledescription:
INFO:  sampletype              mp4a
INFO:  length                  188393000.00
INFO:  timescale              25000.00
INFO:  language                eng
INFO: sampledescription:
INFO:  sampletype              avc1
7157.616 kB / 48.72 sec (0.6%)


DLC 07-28-2014 02:05 PM

Re: Cannot capture ITV streams


 
Could somebody help me capturing this stream https://www.itv.com/itvplayer/lorraine/22-07-2014? I got the command line but it doesn't seem to work at all:

Code:

rtmpdump -r "rtmpe://cp72511.edgefcs.net:1935/ondemand" –a "ondemand?auth=eaEcPcscNahaPambIbbbpakdfaJdFaFardga9aFb.aBcFdLatbRcRabcncvdbdadvc0-bt1P3D-eS-5zu3LAujwryq&aifp=v002&slist=production –W "https://www.itv.com/mediaplayer/ITVMediaPlayer.swf" -p "https://www.itv.com/itvplayer/lorraine/22-07-2014" -y "mp4:production/priority/rtmpecatchup/1200/9F/1-9360-1184-001/1-9360-1184-001_lorraine-(2014)_tx220714_itv1200_16x9-reg.mp4" -o "Lorraine.flv"


All times are GMT -6. The time now is 04:13 AM.