m6replay.fr: rtmpdump error: RTMP_ReadPacket, failed to read RTMP packet header

[John Smith]

Hello,

I have used rtmpdump with rtmpsrv last week without problem.
This morning, I am unable to use it because I am getting this error:
ERROR: RTMP_ReadPacket, failed to read RTMP packet header

I can however view the video without problem with my browser.

Can someone have an hint on the problem ?

Thanks !

Here is the verbose output:

rtmpdump -V -r "rtmpe://groupemsix.fcod.llnwd.net:1935/a2883/d1/" -a "a2883/d1/" -f "LNX 10,0,45,2" -W "http://groupemsix.vo.llnwd.net/o24/u/replayrelease/libs/StatPlaylibrary2.swf" -p "http://www.m6replay.fr/#/emissions/top-chef/9660" -y "mp4:production/regienum/m6_top-chef_30284_150320102040.mp4" -o m6_top-chef_30284_150320102040.mp4
RTMPDump v2.2a
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Parsing...
DEBUG: Parsed protocol: 2
DEBUG: Parsed host    : groupemsix.fcod.llnwd.net
DEBUG: Parsed app     : a2883/d1
DEBUG: Protocol : RTMPE
DEBUG: Hostname : groupemsix.fcod.llnwd.net
DEBUG: Port     : 1935
DEBUG: Playpath : mp4:production/regienum/m6_top-chef_30284_150320102040.mp4
DEBUG: tcUrl    : rtmpe://groupemsix.fcod.llnwd.net:1935/a2883/d1/
DEBUG: swfUrl   : http://groupemsix.vo.llnwd.net/o24/u/replayrelease/libs/StatPlaylibrary2.swf
DEBUG: pageUrl  : http://www.m6replay.fr/#/emissions/top-chef/9660
DEBUG: app      : a2883/d1/
DEBUG: flashVer : LNX 10,0,45,2
DEBUG: live     : no
DEBUG: timeout  : 120 sec
DEBUG: SWFSHA256:
94 75 85 0D 5F 9E B1 12 92 02 9B 6F 3D C6 94 18 C8 7D 28 E7 3F 89 6E 44 A4 7D 85 CC E6 2C FA A3 
DEBUG: SWFSize  : 33672
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
DEBUG: RTMP_Connect1, ... connected, handshaking
DEBUG: HandShake: Client type: 06

DEBUG: HandShake: DH pubkey position: 472
DEBUG: HandShake: Client digest offset: 1383
DEBUG: HandShake: Initial client digest: 
7E DE E0 3C 3A 75 68 94 77 1A 2B 5A 06 95 94 86 73 E9 65 F4 9B 3C 0F 60 A4 8D 8F 0A 50 5B 8D 7C 
DEBUG: HandShake: Type Answer   : 08
WARNING: HandShake: Type mismatch: client sent 6, server answered 8
DEBUG: HandShake: Server Uptime : 1775321601
DEBUG: HandShake: FMS Version   : 3.5.2.1
DEBUG: HandShake: Server DH public key offset: 49
DEBUG: HandShake: Secret key: 
7D D5 56 7B EE F0 60 7D 5A A6 A4 02 F8 EF F8 BF 52 6A 2D 96 63 93 8D 0E A3 89 EB DA 24 EF 41 E1 AA 55 90 1E C8 ED CB AF 3F A5 F9 5B 07 C3 CF B9 40 F8 04 20 4D B2 91 0B 02 16 51 D7 57 21 DF 13 4F 5F 61 D8 A9 CC C0 AF 8B 38 B1 C9 C2 47 3D 23 78 66 48 4E 73 98 B8 BB 66 75 ED 56 77 2A 18 BD 98 55 41 0C C1 FD 6A 6B 0B 03 15 55 70 A2 67 19 D1 71 40 D3 39 B0 B3 CD 13 C1 3C 16 77 0E E1 68 
DEBUG: RC4 Out Key: 
B5 58 4D 8D 17 1B 43 2D 80 05 67 FE FC 55 D8 21 
DEBUG: RC4 In Key: 
2C BF C0 37 06 64 3C F6 A1 03 E8 BB 46 48 A6 9C 
DEBUG: HandShake: Calculated digest key from secure key and server digest: 
58 DC 6B 30 70 81 BD 42 31 4A 42 93 51 F5 69 C7 88 CD D7 83 81 56 70 16 62 6B F8 0B E5 33 2F 7D 
DEBUG: HandShake: Client signature calculated:
79 CB 04 71 57 B2 BE 3B B2 3A 10 16 CA B8 65 52 3F 30 5A 49 12 CE 66 4D 94 D4 C7 57 56 4A 5D F4 
DEBUG: HandShake: Server sent signature:
C5 8A 41 C5 DB D5 80 DD 65 75 5A 2B 9C AA 30 83 2B 40 04 D8 D3 30 03 EC D6 5A 70 4E 1A 33 2D DD 
DEBUG: HandShake: Digest key: 
F0 A1 F5 DA 4D D3 2D 81 C4 5A 4C FC 38 D9 93 B1 92 9D 1C DD 16 24 9A FB 18 88 F9 C3 E8 82 3B 20 
DEBUG: HandShake: Signature calculated:
C5 8A 41 C5 DB D5 80 DD 65 75 5A 2B 9C AA 30 83 2B 40 04 D8 D3 30 03 EC D6 5A 70 4E 1A 33 2D DD 
DEBUG: HandShake: Genuine Adobe Flash Media Server
DEBUG: HandShake: Handshaking finished....
DEBUG: RTMP_Connect1, handshaked
DEBUG: Invoking connect
INFO: Connected...
DEBUG: HandleServerBW: server BW = 2500000
DEBUG: HandleClientBW: client BW = 2500000 2
DEBUG: HandleCtrl, received ctrl. type: 26, len: 3
DEBUG: HandleCtrl, SWFVerification ping received: 
DEBUG: sending ctrl. type: 0x001b
DEBUG: Sending SWFVerification response: 
00 1B 01 01 00 00 83 88 00 00 83 88 F5 BE DE 4D 98 5A 10 DF 54 1A CE 42 34 01 87 14 83 97 3E B0 C9 C5 4F FA BC 6A 24 ED 0B FB 9E 53 
DEBUG: RTMP_ClientPacket, received: invoke 240 bytes
DEBUG: (object begin)
DEBUG: (object begin)
DEBUG: Property: <Name:                    fmsVer, STRING:	FMS/3,5,3,837>
DEBUG: Property: <Name:              capabilities, NUMBER:	127.00>
DEBUG: Property: <Name:                      mode, NUMBER:	1.00>
DEBUG: (object end)
DEBUG: (object begin)
DEBUG: Property: <Name:                     level, STRING:	status>
DEBUG: Property: <Name:                      code, STRING:	NetConnection.Connect.Success>
DEBUG: Property: <Name:               description, STRING:	Connection succeeded.>
DEBUG: Property: <Name:            objectEncoding, NUMBER:	0.00>
DEBUG: Property: <Name:                      data, OBJECT>
DEBUG: (object begin)
DEBUG: Property: <Name:                   version, STRING:	3,5,3,837>
DEBUG: (object end)
DEBUG: (object end)
DEBUG: (object end)
DEBUG: HandleInvoke, server invoking <_result>
DEBUG: HandleInvoke, received result for method call <connect>
DEBUG: sending ctrl. type: 0x0003
DEBUG: Invoking createStream
DEBUG: RTMP_ClientPacket, received: invoke 21 bytes
DEBUG: (object begin)
DEBUG: Property: NULL
DEBUG: (object end)
DEBUG: HandleInvoke, server invoking <onBWDone>
DEBUG: Invoking _checkbw
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
DEBUG: Closing connection.

[elch]

Same problem here. Even though I am not from France, it worked perfectly a few weeks ago. Then they started to remove some crucial XML files and now it's not even possible to get information like the playpath easily. See also my thread other thread on Ubuntu-fr.org: http://forum.ubuntu-fr.org/viewtopic...09369#p3309369

I hope they are "fixing" this problem because depending on the RTL.nl stream ("http://stream.rtl.nl/tvistream01") just to watch Nouvelle Star every Wednesday is a bit harsh. (The quality sucks too). I can't understand why they are so restrictive. Those streaming sites are the perfect opportunity to spread the French language and to attract a crowd of highly interested language learners.

Are there any other French sites you're aware of?

[elch]

Interstingly they kept the encrypted XML file with the playpaths untouched ("http://www.m6replay.fr/catalogue/catalogueWeb2.xml"). I've already tried to decrypt the Flash file to get the algorithm but "flare" couldn't extract anything useful.

[John Smith]

Hello,

I had a look at the rtmpdump code, and the events leading to the bug are:
1) invoking onBWDone leads to invoke _checkbw
2) when reading the returned packet, the packet size is exactly zero (r->m_sb.sb_size = 0 and RTMPSockBuf_Fill(&r->m_sb) < 1)

So it seems that wether the sent packet is malformed or the server detect something wrong and doesn't send any information back.

Anyway, I managed to download the stream by removing the "-W url". I absolutely have no idea why it works like this (it makes no sense at all !).

Maybe you can try with this "trick" ?

If you don't have the XML file for finding the correct path, you can use rtmpsrv as described here:
http://forum.ubuntu-fr.org/viewtopic...09369#p3309369

[elch]

This command works alright: (Thanks John!)

rtmpdump -r rtmpe://groupemsix.fcod.llnwd.net:1935/a2883/d1/ -y mp4:production/regienum/m6_top-chef_30284_150320102040.mp4 -o m6_top-chef_30284_150320102040.mp4

Regarding the catalog issue: I think "http://www.m6replay.fr/catalogue/catalogueWeb2.xml" is Base64. But decoding it gives a string which is not really useful to me. Perhaps it's also encrypted somehow?

[elch]

I have sniffed the response of the download.php file. They are using a hash which is only valid for a few seconds. Therefore I couldn't use wget to get that file.

GET /download.php?file=configurationm6_v3.xml&time=[...]&hash=[...]&msg=%E7a%20farte%20? HTTP/1.1

Host: _www.m6replay.fr

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 115

Connection: keep-alive



HTTP/1.1 200 OK

Server: Apache

Content-Type: text/html

Content-Length: 5227

Date: Sun, 21 Mar 2010 18:07:16 GMT

X-Varnish: 2029656810 2029656378

Age: 4

Via: 1.1 varnish

Connection: keep-alive



......<config>
.<services>
..<service name="GetEnCryptCatalogueService">
...<nom>getEnCryptCatalogue</nom>
...<url>http://www.m6replay.fr/catalogue/catalogueWeb3.xml</url>
...<param/>
..</service>
..<service name="SendVideoToFriendService">
...<nom>setUsertoFriend</nom>
...<url>http://www.m6replay.fr/pages/envoyer-a-un-ami-video.php</url>
...<param method="GET">
....<id type="int"/>
....<idSeq type="int"/>
....<url_sem type="string"/>
....<emailTo type="string"/>
....<emailFrom type="string"/>
....<message type="string"/>
....<typeMail type="string"/>
...</param>
..</service>
..<service name="SetRatingService">
...<nom>setRating</nom>
...<url>http://www.m6replay.fr/pages/noter-une-video.php</url>
...<param method="GET">
....<id type="int"/>
....<note type="int"/>
...</param>
..</service>
..<service>
...<nom>setContact</nom>
...<url>http://www.m6replay.fr/pages/contact.php</url>
...<param method="GET">
....<nom type="string"/>....
....<emailFrom type="string"/>
....<sujet type="string"/>
....<message type="string"/>
...</param>
..</service>
..<service name="GetSearchResultService">
...<nom>getSearchResult</nom>
...<url>http://www.m6replay.fr/pages/rechercher.php</url>.
...<param method="GET">
....<service type="string"/>
....<query type="string"/>
...</param>...
..</service>
..<service name="SetVueService">
...<nom>setVue</nom>
...<url>http://www.m6replay.fr/pages/ajouter-vue-video.php</url>
...<param method="GET">
....<id type="int"/>
...</param>...
..</service>
..<service name="SendHotSpotToFriendService">
...<nom>setUsertoFriendHotSpot</nom>
...<url>http://www.m6replay.fr/pages/envoyer-a-un-ami-video.php</url>..
...<param method="GET">
....<id type="int"/>
....<idSeq type="int"/>
....<url_sem type="string"/>....
....<emailTo type="string"/>
....<emailFrom type="string"/>
....<message type="string"/>
....<typeMail type="string"/>
...</param>
..</service>
.</services>
.<path>..
..<m6fr>http://www.m6.fr/</m6fr>
..<image>http://images.m6replay.fr</image>..
..<rss>http://app.m6replay.fr/rss/rssflash.xml</rss>
..<footerRef>http://www.m6replay.fr/replay/footer-new.html</footerRef>
..<faq>http://app.m6replay.fr/static/faq.html</faq>
..<cgu>http://app.m6replay.fr/static/cgu.html</cgu>
..<contact>http://app.m6replay.fr/m6replay/jsp/contact.jsp</contact>
..<envoiami>http://app.m6replay.fr/m6replay/jsp/envoi.jsp</envoiami>
..<blog>http://blogm6replay.m6blog.fr/</blog> ..
..<swf>
...<root>http://groupemsix.vo.llnwd.net/o24/u/production/swf/</root>...
...<estatLib>eStatNetStream_080709.swf</estatLib>
..</swf>
.</path>
.<estat>
.<playerid>269052198344</playerid>
.</estat>
.<!-- Light publicity on home page, comment to hide

.<pub_light_home>
..<path>http://groupemsix.vo.llnwd.net/o24/u/replayrelease/skins/</path>
..<filename>nc_light.swf</filename>
..<position_x>525</position_x>
..<position_y>505</position_y>
.</pub_light_home>
-->
.<stream>
. <servername>groupemsix.fcod.llnwd.net</servername>
. <application>/a2883/d1/</application>
.</stream>
.<title_prefix><![CDATA[M6 Replay : ]]></title_prefix>
.<nav_missing>
..<item pos="0">
...<title><![CDATA[Choisir par genre]]></title>
...<label><![CDATA[ACCUEIL]]></label>
..</item>
..<item pos="5">
...<title><![CDATA[Choisir par horaire]]></title>
...<label><![CDATA[CHOISIR PAR HORAIRE]]></label>...
..</item>
.</nav_missing>
.<panel_top>
..<item id="0">
...<label><![CDATA[LES PLUS VUES]]></label>
..</item>
..<item id="1">
...<label><![CDATA[DERNIERE CHANCE]]></label>
..</item>
.</panel_top>
.<footer>
..<item name="copyright">
...<label><![CDATA[.. M6web 2009]]></label>
..</item>
..<item name="help" estattag="FAQ">
...<label><![CDATA[Aide]]></label>
...<url type="popup"><![CDATA[http://www.m6replay.fr/static/faq.html]]></url>
..</item>
..<item name="cgu" estattag="CGU">
...<label><![CDATA[CGU]]></label>
...<url type="popup"><![CDATA[http://www.m6replay.fr/static/cgu.html]]></url>
..</item>
..<item name="contact" estattag="Contact">
...<label><![CDATA[Nous contacter]]></label>
...<url type="popup"><![CDATA[http://www.m6replay.fr/pages/contact.php]]></url>
..</item>
..<item name="blog">
...<label><![CDATA[Blog]]></label>
...<url type="_blank"><![CDATA[http://blogm6replay.m6blog.fr/]]></url>
..</item>
..<item name="rss" estattag="RSS">
...<label><![CDATA[RSS]]></label>
...<url type="_blank"><![CDATA[http://www.m6replay.fr/rss/rssflash.xml]]></url>
..</item>
..<item name="join">
...<label><![CDATA[Nous rejoindre]]></label>
...<url type="_blank"><![CDATA[http://www.groupem6.fr/index.php/m6/Human-Ressources]]></url>
..</item>
..<item name="corporate">
...<label><![CDATA[Le groupe M6]]></label>
...<url type="_blank"><![CDATA[http://www.groupem6.fr/]]></url>
..</item>
.</footer>
.<header>
..<item name="chaine">...
...<url type="_blank"><![CDATA[http://www.m6.fr]]></url>
..</item>
..<item name="m6replay" >...
...<url type="_blank"><![CDATA[http://www.m6replay.fr]]></url>
..</item>
..<item name="w9replay" >...
...<url type="_blank"><![CDATA[http://www.w9replay.fr]]></url>
..</item>
..<item name="m6bonus" >
...<url type="_blank"><![CDATA[http://www.m6bonus.fr]]></url>
..</item>
..<item name="m6vod">...
...<url type="_blank"><![CDATA[http://www.m6vod.fr]]></url>
..</item>
.</header>
</config>

The following part is interesting:

.<services>
..<service name="GetEnCryptCatalogueService">
...<nom>getEnCryptCatalogue</nom>
...<url>http://www.m6replay.fr/catalogue/catalogueWeb3.xml</url>
...<param/>
..</service>

Obviously catalogueWeb2.xml is outdated because they now refer to catalogueWeb3.xml in this file.

Unfortunately, catalogueWeb3.xml is also encrypted.

[John Smith]

Ok I begin to understand rtmpdump a little better.
Removing the "-W url" remove the SWFVerification, so although you got temporarily access to the stream, you will be interrupted very soon.
So I keep having to resume the download dozens of times, not very practical.

I'll try to help the RTMPdump hackers on the mailing-list and keep this forum informed.

[John Smith]

It seems that it's related to the Flash 10 handshake.
A totally new SWF hash method has been introduced which will take more work to decode.
Follow-up in the rtmpdump mailing-list for those interested.

[elch]

The number after the "catalogueWeb" seems to stand for the month because they've already added a catalogueWeb4.xml file which makes sense.

[elch]

I have progressed a bit:

We could use mChicheReplay which works fine but is very bloated and only runs on Windows. I've already tried Mono and Wine. Nothing worked.

But I could get useful information through disassembling the binary. (although I didn't get any code only function names and a few variables)

wget http://www.m6replay.fr/catalogue/catalogueWeb4.xml
base64 -d catalogueWeb4.xml > out

From mChicheReplay.exe.config:

            <setting name="K1" serializeAs="String">
                <value>ODdKR0gxa09KZ2hrajA3NzdiajMyNlBvVmhn</value>
            </setting>
            <setting name="K2" serializeAs="String">
                <value>RWxGc2cuT3Q=</value>
            </setting>

$ echo ODdKR0gxa09KZ2hrajA3NzdiajMyNlBvVmhn | base64 -d
87JGH1kOJghkj0777bj326PoVhg

$ echo RWxGc2cuT3Q= | base64 -d
ElFsg.Ot

The crypto algorithm for the XML file is Blowfish ECB.

$ openssl bf-ecb -d -in out -out out.dec

I tried both keys, none of them worked.

The developer also included swfmill, no idea why. Perhaps K1 and K2 are updated regularly and these values are hidden somewhere within it.

Hints and suggestions are appreciated. I've now spent over two hours and did not really get further. Thanks!