Question: 4od, rtmpsuck, cookies and authentication

[Mossy]

Two questions:
1) Why does rtmpsuck sometimes not record all parameters from the rtmp url?

2) How to handle websites that require you to login before you watch videos, and use cookies to store login information

Using
Linux 3.2.0-32-generic
rtmpdump/librtmp 2.4~20110711.gitc28f1bab-1

Note: This entire post relates only to Channel4/4od videos from before August. Since sometime mid-August all new videos use Http Dynamic Streaming on that site.

I'm using librtmp in a python script (XBMC plugin) to stream videos from 4od. There are two different servers that the videos stream from, with slightly different authentications, one is ll.securestream.channel4.com the other is ak.securestream.channel4.com. When you stream a video it randomly comes from one of those servers. Until recently both worked fine from the plugin, but since the other day the AK server has stopped working. I think they started requiring users to be logged in to watch videos older than 30 days at the same time.

In order to figure out what's changed I've been using rtmpsuck to compare the rtmp parameters from the plugin and from the browser. The parameters look identical in that comparison, but for the LL server the authentication details are missing from app and tcUrl in the rtmpsuck output, even though the videos play.

librtmp parameters when using LL server for video at http://www.channel4.com/programmes/f...er/4od#3188820

rtmpe://ll.securestream.channel4.com:1935/a4174/e1/?e=1351863246&h=1d729ce414034a9a73c91bc4f01c4615 playpath=mp4:xcuassets/CH4_08_02_900_42045006001002_002.mp4?e=1351863246&h=1d729ce414034a9a73c91bc4f01c4615 swfurl=http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf flashver=LNX\2011,2,202,243 pageurl=http://www.channel4.com swfvfy=true

rtmpsuck output when using LL server:

RTMP Proxy Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
Processing connect
app: a4174/e1
flashVer: LNX 11,2,202,243
swfUrl: http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf
tcUrl: rtmpe://ll.securestream.channel4.com:1935/a4174/e1
pageUrl: http://www.channel4.com
WARNING: HandShake: Type mismatch: client sent 6, server answered 8

The fact there is parameter data missing from rtmpsuck's output when using LL means that the browser could be truncating data from AK.

For comparison here's the info for AK.
librtmp parameters:

rtmpe://ak.securestream.channel4.com:1935/4oD/?auth=da_cqbuaobgcEb8bkc_dMbucCdzdvcWdxcv-bqK6.e-eS-jyM-oRpbj8jfkXmdr8jbsVockfsVnUkgqSj7ndpfkSjdjbmcrakSraoekfsXmekeqSjbo1qfkekXkdndsfkh&aifp=v002&slist=assets/CH4_08_02_900_42045005001002_002.mp4 playpath=mp4:assets/CH4_08_02_900_42045005001002_002.mp4?auth=da_cqbuaobgcEb8bkc_dMbucCdzdvcWdxcv-bqK6.e-eS-jyM-oRpbj8jfkXmdr8jbsVockfsVnUkgqSj7ndpfkSjdjbmcrakSraoekfsXmekeqSjbo1qfkekXkdndsfkh&aifp=v002&slist=assets/CH4_08_02_900_42045005001002_002.mp4 swfurl=http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf flashver=LNX\2011,2,202,243 pageurl=http://www.channel4.com swfvfy=true

rtmpsuck output for AK:

RTMP Proxy Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
Processing connect
app: 4oD?auth=da_cOdKdya9cNdnbjddaaaHataPd6cGdxdP-bqK6hd-eS-dAS-oUkgsfnWscm5j9nUs9lYr9mXmclVjeqYn7k9tYnTtVnRjcm8s9k8rclfm8lTkXqTn6ldsdnRtfn3j9nT&aifp=v002&slist=assets/CH4_08_02_900_42045005001002_002.mp4
flashVer: LNX 11,2,202,243
swfUrl: http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf
tcUrl: rtmpe://ak.securestream.channel4.com:1935/4oD?auth=da_cOdKdya9cNdnbjddaaaHataPd6cGdxdP-bqK6hd-eS-dAS-oUkgsfnWscm5j9nUs9lYr9mXmclVjeqYn7k9tYnTtVnRjcm8s9k8rclfm8lTkXqTn6ldsdnRtfn3j9nT&aifp=v002&slist=assets/CH4_08_02_900_42045005001002_002.mp4
pageUrl: http://www.channel4.com
WARNING: HandShake: Type mismatch: client sent 6, server answered 8

Note that app and tcUrl retain the authentication info passed to librtmp.

Is there a way that I can be sure I'm getting all the information passed to the rtmpe server?

Output in XBMC, using AK:

14:31:20 T:2975841088    INFO: Parsing...
14:31:20 T:2975841088    INFO: Parsed protocol: 2
14:31:20 T:2975841088    INFO: Parsed host    : ak.securestream.channel4.com
14:31:20 T:2975841088    INFO: Parsed app     : 4oD?auth=da_cec.dCczdzdUcpbibXbvb4cqbBdYcmbY-bqK9KP-eS-hzO-rfo8obqTpRsStTkfqXp7rVkVl9j8oSlesWpXo8qVpTrdsekapbqRqakWl9jandmVr8oap8pgp5sWtblX&aifp=v002&slist=assets/CH4_08_02_900_42045006001002_002.mp4
14:31:20 T:2975841088   ERROR: RTMP_Connect0, failed to connect socket. 111 (Connection refused)

Note also that the above librtmp parameters for AK was done with some of parameters changed slightly to match exactly what rtmpsuck showed was coming from the browser.

The following is are the librtmp parameters as they were previously, which I believe was working up until a few days ago:

rtmpe://ak.securestream.channel4.com:1935/4oD/?ovpfv=1.1&auth=da_bcbkbNaVbDdecDdabMdNb2bkb0c2b9d4-bqK9XP-eS-fyM-tSqetWncscj7r8tWnejemVnflVpfl9jctRrTsbn8tUkVr8tToSkVl9oWlYp6lejftYrcscnethkXs0se&aifp=v002&slist=assets/CH4_08_02_16_49122006001001_002.mp4 playpath=mp4:assets/CH4_08_02_16_49122006001001_002.mp4?auth=da_bcbkbNaVbDdecDdabMdNb2bkb0c2b9d4-bqK9XP-eS-fyM-tSqetWncscj7r8tWnejemVnflVpfl9jctRrTsbn8tUkVr8tToSkVl9oWlYp6lejftYrcscnethkXs0se&aifp=v002&slist=assets/CH4_08_02_16_49122006001001_002.mp4 swfurl=http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf swfvfy=true

... and the librtmp output:

Full text, ending with ...

NetConnection.Connect.Rejected
AccessManager.Reject ] : Access denied!

The differences between the librtmp outputs may indicated the the changes I made to the parameters (in an attempt to match what was being passed from the browser) made things worse.


Question 2)
It's possible that only users who are logged in can view videos from the AK server, in which case I need to know how to handle the cookies that are used to store login/session data. Obviously rtmp doesn't know anything about cookies, so how is this normally handled by websites? Do they usually only allow to rtmp from IPs of users who are logged in? If that is the case with AK then when I'm logged in to the site in a browser, I would expect to be able to stream with rtmpdump. That does not happen. Or is it normally the case that having the correct rtmp parameters is enough to allow access, regardless of how the website handles logins?

[svnpenn]

Originally Posted by Mossy

librtmp parameters when using LL server for video at http://www.channel4.com/programmes/f...er/4od#3188820

rtmpe://ll.securestream.channel4.com:1935/a4174/e1/?e=1351863246&h=1d729ce414034a9a73c91bc4f01c4615 playpath=mp4:xcuassets/CH4_08_02_900_42045006001002_002.mp4?e=1351863246&h=1d729ce414034a9a73c91bc4f01c4615 swfurl=http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf flashver=LNX\2011,2,202,243 pageurl=http://www.channel4.com swfvfy=true

rtmpsuck output when using LL server:

RTMP Proxy Server v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
Processing connect
app: a4174/e1
flashVer: LNX 11,2,202,243
swfUrl: http://www.channel4.com/static/programmes/asset/flash/swf/4odplayer-11.31.2.swf
tcUrl: rtmpe://ll.securestream.channel4.com:1935/a4174/e1
pageUrl: http://www.channel4.com
WARNING: HandShake: Type mismatch: client sent 6, server answered 8

Firstly you have truncated the RtmpSuck output. The "playpath" data comes few lines after the "pageurl". Secondly RtmpSuck does most of it work "behind the scenes". It only prints parameters that have been explicitly coded. In fact the UsherToken, or "-j" does not even print on official master, only on my build and KSV build.

github.com/svnpenn/rtmpdump/commit/6ed7

[Mossy]

Thanks for the information. So to capture all the data that's passed to the rtmp server, I should use something RtmpExplorer?

Originally Posted by svnpenn

Firstly you have truncated the RtmpSuck output.

Not really. The rtmpsuck output repeats everything from the Processing connect line to the WARNING: HandShake: line in an infinite loop until rtmpsuck is stopped. rtmpsuck continues this output even after the browser is closed. I don't know if that's normal or not, but if I'm missing lines after pageUrl then somethings going wrong somewhere. Any ideas of how to fix that, or how to narrow down where the problem is? I get the same results using your version of the build.

[svnpenn]

Originally Posted by Mossy

The rtmpsuck output repeats everything from the Processing connect line to the WARNING: HandShake: line in an infinite loop until rtmpsuck is stopped.

You are only going to get a playpath if you are doing a proper port redirect of RTMP traffic. On Linux this can be done by either editing the hosts file or using iptables

github.com/svnpenn/rtmpdump/blob/svnpenn/README

$ rtmpsuck
RTMP Proxy Server v2.4-49-g6ed7d74
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
Processing connect
app: a2246/o23/?_fcs_vhost=wbworldtv.fcod.llnwd.net
flashVer: WIN 11,4,402,278
swfUrl: http://pdl.warnerbros.com/cwtv/digital-smiths/production_player/vsplayer
.swf
tcUrl: rtmpe://68.142.121.230/a2246/o23/?_fcs_vhost=wbworldtv.fcod.llnwd.net
pageUrl: http://cwtv.com/cw-video/americas-next-top-model/the-girl-who-becomes-a
rt-for-tyra/?play=2f5f6874-9857-4a4c-bc06-afe17f1f5929
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
Playpath: mp4:cwtv/videos/2012/10/26/ATM1910-The-Girl-Who-Becomes-Art-For-Tyra_b
6179fd0e_500kbps.mp4

[Mossy]

I have been doing the following, which I believe is a proper port redirect:

sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT

[svnpenn]

Originally Posted by Mossy

I have been doing the following, which I believe is a proper port redirect:

sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner root -j REDIRECT

This means that you either havent tried editing the hosts file, or you did, and didnt mention it. Either way you are wasting my time.