Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)

- Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)

- -

#EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

(http://stream-recorder.com/forum/showthread.php?t=22946)

streamcrank

04-30-2017 01:34 PM

#EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

It's called HTTP Live Streaming and it's a standard..

There is usually a master m3u8 with, for example different resolutions or audio only. Then each version of the content has it's own m3u8 that has the ts files you download and copy(copy *.ts example.mp4 on Windows for example). Then run it through ffmpeg to clean it up.

If it uses encryption then each .ts has it's own crypto key url("#EXT-X-KEY:METHOD=AES-128,URI=") and iv(example:"IV=0x00000000000000000000000000000007" ) that is used to decrypt it. The key is the body of the response of that http GET to the key url. Each key is for each TS there is no inheritor or weird padding or mixing going on..

How big networks make it more secure: The spec allows for a global key also in the ts-m3u8 that is used to decrypt each ts key, and the hosts usually timeout sessions on these and only server over https/tls. FFMPEG can still beat this security if you pass it a fresh enough URL. Sometime you have to manually dump using a cookie but not often. Grab the page with python and use the cookie header to grab ts files and either decrypt with a python encryption library(no native library) or use openssl CLI. Some might start using HTML5 storage too. With spec security just use the following.

How to easily get the m3u8:

Code:

//F12 then in console:

document.getElementsByTagName('video')[0]

How to rip even the secure ones(since HTML5 player has strict rules on key exchange). NOTE: if FFMPEG stops for any reason use the -ss option in front of input to resume ripping to a new file and just concat later; this happens with me with 1080 stream ripping.

Code:

ffmpeg -protocol_whitelist file,udp,rtp,crypto,tcp,https,rtsp,tls -i "TS-M3U8 URL HERE" -c:a copy -c:v copy "OUTPUT FILE HERE.ts"

Best transcode

Code:

//remove -hwaccel auto if you get errors

ffmpeg.exe -hwaccel auto -i "TS FILE HERE" -c:a copy -c:v x265 "OUTPUT PATH HERE.mp4"

P.S. I only got in to streaming a couple days ago but have seen this before on paid podcast networks except not encrypted.. It's easy.. If you use it I suggest tighter rulers on the global key. THE future: Using TPM hardware for decryption with binding to HTML5 video internals.

kingstaytheking

05-07-2017 06:24 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

this isn't gonna work for encrypted streams .. like hbogo, showtime anytime etc...

streamcrank

05-09-2017 04:46 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by kingstaytheking (Post 91493)

this isn't gonna work for encrypted streams .. like hbogo, showtime anytime etc...

If it works on Uplynk HLS streams it'll work on all of them. Uplynk uses master+segment keys in AES-128.. I feed them directly to FFMPEG and get clean 1080p rips..

Unless they use a custom player with FlashPlayer they have to comply with the HLS standard to get it to work with HTML5 <video>.. Most browsers shut off FlashPlayer based on domain policy or MAC so HTML5 is pretty much their only option.. Third party players like JWPlayer are even defaulting to HLS standard now.. All of their developers have to be force fed knowledge like most textbook CS grad software people so don't expect them to tighten security around the key URIs anytime soon..

arec	05-09-2017 10:44 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

1 Attachment(s)

This is the kind of stream I am having difficulty with. When I try your code...

ffmpeg -protocol_whitelist file,udp,rtp,crypto,tcp,https,rtsp,tls -i "TS-M3U8 URL HERE" -c:a copy -c:v copy "OUTPUT FILE HERE.ts"

using my own filenames I get an error about "unable to open key file". I am not sure how to get that key file from the m3u8 file. I have read that it could be found in a cookie but also that it could be found in them m3u8 file itself.

streamcrank

05-09-2017 02:07 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by arec (Post 91543)

second red text is from first red text. They likely use cookies or referrer or timeout or all of the above to protect individual and/or master key URI. With these you need to use a python script the uses cookie header and sets referrer header. This isn't an uplynk thing it's a client thing.

arec	05-09-2017 06:12 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Roger that, thank you steamcrank!

kingstaytheking

05-10-2017 09:34 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

I still don't buy it

I will share my personal hbogo account with you .. if you get streams working I'll even donate you some money

but I just don't see it working

contact me if interested

streamcrank

05-13-2017 03:05 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by kingstaytheking (Post 91565)

I still don't buy it

I will share my personal hbogo account with you .. if you get streams working I'll even donate you some money

but I just don't see it working

contact me if interested

I'll do it if the donation is bitcoin. I have to go through a proxy or VPN that isn't blacklisted by HBO because I'm outside the US right now.

Unless your target site is using a flash player they're forced to stick to the standards supported by the target browser HTML5 <video> support. There is no if and or but.. Both HTML5 DRM and HLS use clear key exchanges only protected by weak session policies. FFMPEG does great with HLS encrypted as long as you have the bandwidth for HD and they don't check cookie and referrer headers on key URIs.

WHOOKIDSON

05-13-2017 03:27 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by streamcrank (Post 91632)

I'm willing to donate as well. Keep us updated. Check your PM.

EDIT:

And btw. HBO and a few others don't use #EXT-X-KEY:METHOD=AES-128,URI=

Last time I checked it was #EXT-X-KEY:METHOD=SAMPLE-AES,URI="faxs://faxs.adobe.com",KEYFORMAT="com.apple.streamingkeyd elivery",KEYFORMATVERSIONS="1",IV=

kingstaytheking

05-13-2017 03:40 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

hbo works with vpn

and sure .. btc is fine

Yelloworanges

05-13-2017 03:47 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by streamcrank (Post 91294)

P.S. I only got in to streaming a couple days ago

Quote:

Originally Posted by WHOOKIDSON (Post 91633)

Last time I checked it was #EXT-X-KEY:METHOD=SAMPLE-AES,URI="faxs://faxs.adobe.com",KEYFORMAT="com.apple.streamingkeyd elivery",KEYFORMATVERSIONS="1",IV=

Nothing streamcrank has said indicates that he is familiar with flash access.

kingstaytheking

05-13-2017 03:50 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by WHOOKIDSON (Post 91633)

crap.. you are right.. it is SAMPLE-AES

I didn't check on this for a while... was pretty sure it's AES-128

but now I see they use fairplay encryption with xs key delivery
faxs:// url points the player to the Adobe Access (XS) system ... key is probably never even passed through the browser

this completely bypasses widevine .. so netflix approach is a no go

maybe somebody out there knows the trick... but with written above... I just don't see it

Yelloworanges

05-13-2017 04:06 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by kingstaytheking (Post 91636)

but now I see they use fairplay encryption with xs key delivery
faxs:// url points the player to the Adobe Access (XS) system ... key is probably never even passed through the browser

this completely bypasses widevine

maybe somebody out there knows the trick... but with written above... I just don't see it

I don't see it either. XS solution is probably closely held unless you've seen it on darkweb. I don't have hbo but is hls with XS the method they use on all device platforms?

kingstaytheking

05-13-2017 04:41 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

that's what I'm wondering too

web one is a no go as far as i know...
but then again... I could be wrong

WHOOKIDSON

05-13-2017 04:51 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

I'll try it on the Xbox One/Roku/Fire TV later on tonight.

They're probably still the same. HBO Now uses dash on the Xbox One and Roku. I'll try the HBO Go tonight.

Yelloworanges

05-13-2017 04:55 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

dash is most commonly widevine but I have seen playready versions of it like on some content on nbcsports.

kingstaytheking

05-13-2017 05:14 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

even if you grab dash link ... u can't do much with it..

key request is gonna be https ... fiddler or charles cert on device will be required to decrypt it .. and "man in the middle" is another topic that will be hard to pull off for something like this

i don't see them encrypting request of video itself.. that's gonna be http .. which means it;s easy to grab .mpd

but key exchange is httpS... 1000000% ... regardless of which device we are talking about

in so many words, it can't be done

WHOOKIDSON

05-13-2017 05:25 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

I already sniffed the DASH stuff (which came from HBO NOW). I was just curious to see what kind of stuff was being used on those devices for that specific app. I was talking about doing HBO Go tonight. Far as ripping HBO goes the easiest way I've seen to rip it is to use the satellite or cable's provider's streaming site/service. I used to rip from XfinityTV and AT&T U-Verse. Last time was late summer of last year. Then I got into AMZN and NF stuff. I might have to dust off some old scripts to see if the old stuff still works for "HBO" (through the provider's login/site).

And more and more sites are blocking the certs from Fiddler and Charles. I remember last year some sites would work with Fiddler but not with Charles and vice versa. I've always had better success with Charles tho.

kingstaytheking

05-13-2017 05:46 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

what are xfinity and att using?

if I'm not mistaking they have on demand content from hbo

WHOOKIDSON

05-13-2017 05:55 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

"Last year" the key was being sent by "#EXT-X-KEY:METHOD=AES-128,URI=" (I'm sure XfinityTV is still using Adobe FlashAccess DRM tho)

I don't know about now. But that is how it was for XfinityTV so all you had to do was download the key and then all of the .ts files and decrypt that way locally. I can't really remember what U-Verse was. I was mainly ripping from their site. Far as U-Verse goes now "some" of their stuff uses DASH (Chrome) and the video can be "decrypted" using the tool. I haven't really had time to play with Xfinity since they did the major upgrade to their app and site.

And yeah, both of them have HBO content.

streamcrank

05-14-2017 02:00 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by Yelloworanges (Post 91635)

Nothing streamcrank has said indicates that he is familiar with flash access.

What is "flash access".. It doesn't exist..

Flash can use custom same-domain protocols and encryption and has a a few codecs because it has ASC which you can code anything. You dump the ASC with a flash decompiler then reverse engineer it.. Again "flash access" does exist. JWPlayer is flash based and I've extracted TEA encryption keys from it before.

AES-SAMPLE works the same as AES-128 except they can change keys instead of just IV per segment. I don't care about these other networks because all the big networks use uplynk and zype and uplynk and zype only use variants of HLS that are all standards compliant..

https://bitmovin.com/tutorials/hls-encryption/

Yelloworanges

05-14-2017 03:10 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by streamcrank (Post 91653)

What is "flash access".. It doesn't exist..

Quote:

Originally Posted by streamcrank (Post 91653)

AES-SAMPLE works the same as AES-128 except they can change keys instead of just IV per segment.

https://bitmovin.com/tutorials/hls-encryption/

Sure, but there is the small matter of actually getting the key in the first place. For HLS implementations of Adobe Access and PlayReady, this is an entirely different proposition than vanilla AES-128 where the key is in plaintext in the first instance.

Quote:

Originally Posted by streamcrank (Post 91653)

I don't care about these other networks because all the big networks use uplynk and zype and uplynk and zype only use variants of HLS that are all standards compliant..

I can't argue with you not caring. Some members here are interested in content that uses actual hls drm methods to protect the key, which is distinct from vanilla AES-128 hls which is not really drm at all.

streamcrank

05-14-2017 03:43 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by Yelloworanges (Post 91656)

I was using the legacy name. As kingstaytheking pointed out earlier, the more precise name is Adobe Access. No flash player need be involved.

https://helpx.adobe.com/adobe-media-...ction-hls.html

As described in the above link, the content can be protected using three modes:
Vanilla
PHLS
Adobe Access 4.0

Sure, but there is the small matter of actually getting the key in the first place. For HLS implementations of Adobe Access and PlayReady, this is an entirely different proposition than vanilla AES-128 where the key is in plaintext in the first instance.

I can't argue with you not caring. Some members here are interested in content that uses actual hls drm methods to protect the key, which is distinct from vanilla AES-128 hls which is not really drm at all.

I've reverse engineered plenty of flash media players and have yet to see one using AdobeAccess. Currently I'm ripping HD streams from a lot of big sites and they all use HLS which is a Apple patent and doesn't include any Adobe spec because it wouldn't work with native <video> HTML5 support. If you see any of the tags I mention in this thread and there is no flash object embedded(so they can have their own custom ASC HLS handling) you can 100% guarantee there is nothing to do with any Adobe spec.. Since most browsers disable FlashPlayer by default they all use HTML5.. Worse Case Scenario: Session timeouts on key URIs with a referrer header check.. Anything else literally breaks playback..

The native HTML5 DRM engine also uses cleartext key exchange over TLS. I haven't seen it used yet but it's well documented.
I'm currently working on a generic tool but FFMPEG is still working when I feed it fresh m3u8. My tool will work the same except I spoof referrer and get a fresh session.

Yelloworanges

05-14-2017 06:41 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by streamcrank (Post 91658)

What a relief. Thanks for clearing that up. I've either been malignantly confused or it has all been in my imagination. I'm forever in your debt.

kingstaytheking

05-14-2017 10:40 AM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by WHOOKIDSON (Post 91645)

hmm tried it... video is decrypted with no problems.. trouble with audio

kingstaytheking

05-14-2017 01:24 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

jutru tired this with AES-128

Code:

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-TARGETDURATION:7

#EXT-X-MEDIA-SEQUENCE:0

#EXT-X-FAXS-CM:long ass key here

#EXT-X-KEY:METHOD=AES-128,URI="faxs://faxs.adobe.com",IV=0x00000000000000000000000000000001

error

Code:

Unable to open key file faxs://faxs.adobe.com

[hls,applehttp @ 00000000026424a0] Error when loading first segment 'http://vod-10.timewarnercable.com/HLS_DRM/MTHM0000000000985623/150000-0-33-376-236128-36-63732-327120.ts'

http://vod-10.timewarnercable.com/HLS_DRM/MTHM0000000000985623/index.m3u8?adId=a0e3c9e5-55b2-4594-aa46-b277bacacc27: Invalid data found when processing input

streamcrank

05-14-2017 02:57 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Anything with "#EXT-X-FAXS-CM" in it is a custom implementation for Adobe Media Server and not HTML5. It uses out of band JS to fetch keys that have tight session restrictions. You can defeat it with custom code the same as with hardened HLS key URI.

kingstaytheking

05-14-2017 03:24 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

if you really have a way.,.. add me to skype
and we can talk

Yelloworanges

05-14-2017 04:22 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by streamcrank (Post 91666)

Setting aside that it doesn't exist, Adobe Primetime for hls can work in a HTML5 player in firefox via the Adobe Primetime Content Decryption Module (CDM).

Dash implementations are largely supplanting hls, but there are html5 players that support Adobe Primetime with hls.

kingstaytheking

05-15-2017 12:05 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

this guy is just talking out of his ass

only reason why any of these protected streams are crackable is widewine bug in chrome .. take that away.. and all premium streams are untouchable

and that's a fact

Code:

ffmpeg -protocol_whitelist file,udp,rtp,crypto,tcp,https,rtsp,tls -i "TS-M3U8 URL HERE" -c:a copy -c:v copy "OUTPUT FILE HERE.ts"

are you kidding me????

why do you even need -protocol_whitelist?

this shit will work for nbc, espn, amc ... and all those streams .. but youtube-dl makes it so easy that is pointless to do any of this

all that shit that you are doing for no reason, replace with
youtube-dl "M3U8 link here"

approach like this
http://stream-recorder.com/forum/sho...71&postcount=3

maybe ...

but name of the this topic.. and than "how to.." explanation is just laughable

I'm out

WHOOKIDSON

05-15-2017 02:41 PM

Re: #EXT-X or m3u8 or TS or AES-128? You got HLS.. Howto Beat it in 2017

Quote:

Originally Posted by kingstaytheking (Post 91691)

Code:

ffmpeg -protocol_whitelist file,udp,rtp,crypto,tcp,https,rtsp,tls -i "TS-M3U8 URL HERE" -c:a copy -c:v copy "OUTPUT FILE HERE.ts"

LOL

I sent him 2 M3U8's 2 days ago to play around with (his request)

Let's see if he can get it done :D

All times are GMT -6. The time now is 10:09 PM.