Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)
-   Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)
-   -  

theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confused.

(http://stream-recorder.com/forum/showthread.php?t=6267)

Confused User 04-05-2010 01:25 PM

theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confused.


 
I am trying to rip a stream. Actually, a whole bunch of streams, but I'll just provide one as an example;

The Stream URL is inside this XML file: "http://release.theplatform.com/content.select?pid=2vaZNBb_CxCP9V5QODRKpTL2vZqvVz_ _"

The streaming server seems to use some sort of time or session based authentication. Every time you refresh that XML file, the URL of the RTMP stream is slightly different.

Nevertheless, if I copy the RTMP stream URL out and paste it in any of the major GUI-based RTMP rippers, they work. I just successfully ripped a number of streams using NetTransport, it works with Replay Media Catcher, etc.

However, I'd like to use RTMPDump, both because it's free and open source, and because since it's a command line utility it'd be very easy to batch automate stuff with it.

I believe the issue is related to the unusual format of the URL; like I said above, this URL is going to expire shortly so you'll have to visit the XML file to get a new, valid URL, but here's the URL from this particular time.

rtmp://cp68811.edgefcs.net/ondemand/?auth=dbEaQdrcOauckcXdCb_b.aqaedta9awaUaj-blUJDK-4q-d9i-3qsFAnKpwE6P9&aifp=1234&slist=Canwest_Broadcast_En tertainment/<break>Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4048.flv

What I notice that's strange is the <break> component. That's very odd for any file system. I tried various ways of switching up the URL (rtmpdump -r url, rtmpdump -r url -y Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4048.flv, etc)

If anyone could visit the XML file, get a valid URL, and successfully figure out a combination of switches necessary to get this working with RTMPDump, that'd be great.

Thanks.

Stream Recorder 04-06-2010 12:50 AM

Re: theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confu


 
An RTMP URL looks like:
rtmp://server/app/playpath

If you have no idea how to extract the server, app and playpath components, you can find them with freeware CooJah.

Confused User 04-06-2010 07:10 AM

Re: theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confu


 
I installed CooJah and ran it on the initial page.

Again, keep in mind that the RTMP url uses some sort of time-sensitive auth so anything I copy/paste will be expired by the time you read it;

Here's what I've got:
Code:

url: rtmp://cp68811.edgefcs.net:1935/ondemand/?auth=dbEcAb.dgcpd2aCcvdEc5dWdLdAcpbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=Canwest_Broadcast_Entertainment/Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4052
app: ondemand/?auth=dbEcAb.dgcpd2aCcvdEc5dWdLdAcpbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=Canwest_Broadcast_Entertainment/
flashVer: WIN 10,0,32,18
pageUrl: http://www.foodnetwork.ca/video/index.html
swfUrl: http://webdata.globaltv.com/global/canwestPlayer/swf/4.1/flvPlayer.swf
tcUrl: rtmp://cp68811.edgefcs.net:1935/ondemand/?auth=dbEcAb.dgcpd2aCcvdEc5dWdLdAcpbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=Canwest_Broadcast_Entertainment/
playPath: Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4052

I can verify that I can download this through Blader. Now I just need to figure out how to transfer this information to RTMPdump.

Now, as far as I can tell, assuming I need 100% of this information, rtmpdump should be called as such:
rtmpdump -r url -a app -p pageUrl -f flashVer -s swfUrl -t tcUrl -y playPath
(obviously, substituting in all of the information above for the variables.)

Here's a direct dump of the results I get running from the command line:
Code:

rtmpdump -r "rtmp://cp68811.edgefcs.net:1935/ondemand/?auth=dbEcAb.dgcpd2aCcvdEc5dWdLdAc
pbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=Canwest_Broadcast_Entertainm
ent/Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4052" -a "ondemand/?auth=d
bEcAb.dgcpd2aCcvdEc5dWdLdAcpbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=C
anwest_Broadcast_Entertainment/" -p "http://www.foodnetwork.ca/video/index.html"
 -f "WIN 10,0,32,18" -s "http://webdata.globaltv.com/global/canwestPlayer/swf/4.
1/flvPlayer.swf" -t "rtmp://cp68811.edgefcs.net:1935/ondemand/?auth=dbEcAb.dgcpd
2aCcvdEc5dWdLdAcpbCc_am-blUY8y-4q-d9i-7onGGpPqAC5O2&aifp=1234&slist=Canwest_Broa
dcast_Entertainment/" -y "Canwest_Broadcast_Entertainment/Food_Glutton_S4_Ep4052
"
RTMPDump v2.2b
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
Starting download at: -0.001 kB
Metadata:
  duration              1296.32
  width                640.00
  height                360.00
  framerate            25.00
  canSeekToEnd          TRUE
  videocodecid          4.00
  videodatarate        480.00
  audiocodecid          2.00
  audiosamplerate      44100.00
  audiosamplesize      16.00
ERROR: Download: Failed writing, exiting!

It clearly detects the duration, width, height, framerate, etc. Much better than it was doing before, which is great. But it still fails. When I throw a -z (to get debug info), it reads what seems like hundreds of packets and then dies. I'll include the last packet or so:

Code:

DEBUG2: RTMP_ReadPacket: fd=380
  0000:  c5                                                .
  0000:  62 80 38 8e 49 ed d1 a9  97 06 69 e7 f5 51 1a e4  b.8.I.....i..Q..
  0010:  8f cb af c5 69 c7 ae f0  76 2f 19 c1 c4 22 c9 48  ....i...v/...".H
  0020:  62 cd 88 1f d9 2e 9d 64  33 76 4e 68 60 4c 2b 81  b......d3vNh`L+.
  0030:  eb 3c f7 32 53 95 8c da  be a1 41 29 94 85 c5 8d  .<.2S.....A)....
  0040:  26 4c da 90 8c 72 bc 06  d8 e2 4f 63 9b e7 ce 88  &L...r....Oc....
  0050:  c2 97 b1 4e 5d fe 6a d2  76 e0 6f 45 95 7d 06 da  ...N].j.v.oE.}..
  0060:  e4 00 9b 4c 4c 7d 4d 00  06 95 f7 df f1 9b e1 32  ...LL}M........2
  0070:  58 a8 4b 40 62 32 75 79  a0 98 7c 07 cb ec f9 b5  X.K@b2uy..|.....
  0080:  d3 00 df ca cd 60 46 46  a4 31 e0 c2 ee 38 7b d6  .....`FF.1...8{.
  0090:  5a 88 5e c7 84 c2 3d 13  19 cf bd 8a af da 3d a3  Z.^...=.......=.
  00a0:  7a ea f9 71 00 ed b5 c0  9f bb 20 61 60 7e c9 01  z..q...... a`~..
  00b0:  8b 14 04 86 52 73 5f e0  da e7 d2 66 a8 f7 7c 98  ....Rs_....f..|.
  00c0:  75 b6 b6 33 34 da 6d 84  9f 38 e8 48 1c d0 ef c1  u..34.m..8.H....
  00d0:  2b 27 f7 1f 7c b2 04 c7  ce 47 f8 59 63 4b 48 44  +'..|....G.YcKHD
  00e0:  e3 d3 c8 d1 79 ac 43 47  37 f0 44 ba 88 f6 6a 97  ....y.CG7.D...j.
  00f0:  38 a6 5b 50 28 3b c7 e8  05 43 be 53 40 5f 11 b1  8.[P(;...C.S@_..
  0100:  6a 1f da ac 57 1f 4f 28  f6 54 aa 13 ff 82 fe f7  j...W.O(.T......
  0110:  c3 6f f5 02 bb 46 44 3e  41 55 9a 77 76 95 4c ec  .o...FD>AU.wv.L.
  0120:  da 3e 66 81 ff b5 09 a2  09 37 86 bd 0b b5 1d 14  .>f......7......
  0130:  cf de 0e c7 45 d7 c2 fc  dd 92 55 f9 16 c4 67 38  ....E.....U...g8
  0140:  ae b7 7f de 51 4d a6 c7  d3 fb 2b 81 79 84 e3 d3  ....QM....+.y...
  0150:  3d 24 fd 11 22 32 d3 ca  dc da c1 bc d2 56 ca a6  =$.."2.......V..
  0160:  5d 53 c1 35 da 43 33 96  29 85 e9 3d 65 a6 0e fd  ]S.5.C3.)..=e...
  0170:  4e f5 ce 30 84 4d f5 c7  e6 5b 4e bd 4b 34 6d 2c  N..0.M...[N.K4m,
  0180:  e2 65 93 60 72 ff 03 57  58 c6 03 fb e9 52 c9 25  .e.`r..WX....R.%
  0190:  01 19 e8 7b c9 18 d7 3c  b0 69 3d 9e e8 19 c3 dd  ...{...<.i=.....
  01a0:  df 71 d0 73 92 9e a5 ff  9a 3e c4 5e 66 fc 04 11  .q.s.....>.^f...
  01b0:  02 de 69 4b 29 77 a7 e4  b9 3a 0e 62 e3 4e 27 91  ..iK)w...:.b.N'.
  01c0:  f1 29 07 8b 5f 45 a6 37  26 c1 95 94 44 ce 58 19  .).._E.7&...D.X.
  01d0:  88 47 1e 8b bb f7 af f5  71 c5 33 ea 28 fd 24 86  .G......q.3.(.$.
  01e0:  0b 6e 2e 4d da 3c 68 71  a7 5f 29 5f aa 35 67 11  .n.M.<hq._)_.5g.
  01f0:  45 1b de a7 d5 9f 60 3e  73 57 d3 b9 ba bd a3 d0  E.....`>sW......
  0200:  e4 c1 1b 91 fc ce bd 9e  a4 dc 72 b2 c5 33 56 c7  ..........r..3V.
  0210:  cf 20 22 9d 6c fc 28 15  ab cb d7 73 2e e2 3a 37  . ".l.(....s..:7
  0220:  3d 2c e9 db 68 41 9b 63  6f 07 16 f4 1f 46 24 ea  =,..hA.co....F$.
  0230:  c0 2e a2 85 6e 35 93 ed  f2 57 a3 5d 72 44 19 78  ....n5...W.]rD.x
  0240:  a7 9c f3 92 0a 08 42 e9  df ef 13 a0 94 0c e5 03  ......B.........
  0250:  b7 ee 23 fb c7 24 3a ad  c9 e4 46 a3 ba 01 4c c3  ..#..$:...F...L.
  0260:  c8 43 53 cc 32 28 21 b2  74 7a 17 55 f2 a8 7f b3  .CS.2(!.tz.U....
  0270:  3b 88 2d 24 e1 88 73 55  a3 21 5c 19 2c 77 19 c4  ;.-$..sU.!\.,w..
  0280:  b8 00 00 00 00 00                                  ......
ERROR: Download: Failed writing, exiting!
DEBUG: Closing connection.

DEBUG2: RTMP_SendPacket: fd=380, size=34
  0000:  43 00 00 00 00 00 22 14                            C.....".
  0000:  02 00 0c 64 65 6c 65 74  65 53 74 72 65 61 6d 00  ...deleteStream.
  0010:  40 14 00 00 00 00 00 00  05 00 3f f0 00 00 00 00  @.........?.....
  0020:  00 00                                              ..
DEBUG: Invoking deleteStream

Instead of throwing a -z, I'll throw a -v to try to get verbosity; If I do that, it seems to just hang:

Code:

RTMPDump v2.2b
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
ERROR: rtmp server sent error
Starting Live Stream

... and it sits there until I ctrl+c the process.

Any further advice? Like I said, downloading the stream clearly works in every GUI tool, I'd just really like to get it working in RTMPdump so I can write external scripts to batch automate this process for me.

Stream Recorder 04-06-2010 07:21 AM

Re: theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confu


 
Quote:

Originally Posted by Confused User (Post 17939)
Code:

ERROR: Download: Failed writing, exiting!

-o parameter?

-W - for SWF URL (if it is required)

Confused User 04-06-2010 08:54 AM

Re: theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confu


 
The issue ended up being not explicitly setting an output file and the download worked fine, thanks for the help.

In fact, I was able to remove the -W, -p, and -f switches. It works fine as -r -a -t -y -o.

But on the other hand, I don't think it was a case of "not reading what was in front of me". I really have no idea why I needed to explicitly set an output file. Without -o, the functionality should cause the stream data to be "written to stdout." according to the manual. There's nothing in the manual or elsewhere suggesting that this would trigger an error state, and the error message doesn't provide any useful diagnostic information that would suggest why it would fail one way and not the other.

Obviously I can still accomplish the same thing writing to a file and feeding that file into any postprocessing rather than dealing directly with the data being pumped into STDOUT, but again there's no indication that I should have to do it that way.

dooglus 11-30-2010 04:30 AM

Re: theplatform.com: RTMP stream works with all GUI rippers, not with RTMPDump, confu


 
I found I only have to specify --host, --app, and --playpath to download Canwest_Broadcast_Entertainment stuff.

Here's a shell script that fetches the URL including the auth token, splits it up and runs rtmpdump:

Code:

#!/bin/sh

pid='C43FtRVCkqcF6nDGMPGudYYK1sa_giDI' # Cesar Millan

start="http://release.theplatform.com/content.select?pid=$pid&UserName=Unknown&Site=global_prg.com&k4=pr&Tracking=True&v4=globalnews&k7=page&v5=programs&v6=16x9&k2=kw&k0=kw&Embedded=True&k8=cnt&Portal=GlobalTV&k3=nk&v3=cbrdcst&k1=kw&v0=ron&v1=programs&v8=news&k6=sck&v2=16x9&TrackBrowser=True&v7=video&k5=ck&TrackLocation=True&k9=cnt&v9=social_issues&Zone=16x9thebiggerpicture&show=16x9_the_bigger_picture&episode=&network=&season=&clipLength=long&format=SMIL"

url=$(wget -o/dev/null -O- "$start" | grep '[.]flv' | cut -d'"' -f2)

host="$(echo "$url" | cut -d/ -f3)"
app="$(echo "$url" | cut -d/ -f4-5 | sed 's/&amp;/\&/g')"
playpath="$(echo "$url" | cut -d/ -f6- | sed -e 's/&lt;break&gt;//' -e 's/[.]flv$//')"
file="$(echo "$playpath" | cut -d/ -f2).flv"

rtmpdump --host "$host" --port 1935 --app "$app/" --playpath "$playpath" -o "$file"

It gets this URL:

Code:

rtmp://cp68811.edgefcs.net/ondemand/?auth=dbEd6ddbdc8aTcOblddb1dYdnaRata3aGby-bm9n.u-4q-d9i-9qoEDpKnBEXN2&amp;aifp=1234&amp;slist=Canwest_Broadcast_Entertainment/&lt;break&gt;Canwest_Broadcast_Entertainment/16X9Nov29-1a.flv
breaks it up as follows:

Code:

host: cp68811.edgefcs.net
app: ondemand/?auth=dbEd6ddbdc8aTcOblddb1dYdnaRata3aGby-bm9n.u-4q-d9i-9qoEDpKnBEXN2&aifp=1234&slist=Canwest_Broadcast_Entertainment
playpath: Canwest_Broadcast_Entertainment/16X9Nov29-1a
file: 16X9Nov29-1a.flv2

and ends up running this command:

Code:

rtmpdump --host cp68811.edgefcs.net --port 1935 --app ondemand/?auth=dbEd6ddbdc8aTcOblddb1dYdnaRata3aGby-bm9n.u-4q-d9i-9qoEDpKnBEXN2&aifp=1234&slist=Canwest_Broadcast_Entertainment/ --playpath Canwest_Broadcast_Entertainment/16X9Nov29-1a -o 16X9Nov29-1a.flv2


All times are GMT -6. The time now is 02:19 AM.