Trying to Download VoD from CBC Sports / CBC Olympics for Rio 2016

Impulse1234 · #1 08-04-2016, 12:16 AM

I've been trying for months to figure out how to download videos from the CBC sports website, to no avail; in particular, that of the upcoming Rio Olympics. I prefer having them downloaded and able to watch at any time instead of having to use slow bandwidth / phone data to stream them, but I've had zero luck being able to download anything off of there in good shape.

Example of something I want to download (first round of women's soccer, Canada vs. Australia): http://olympics.cbc.ca/video/vod/soc...d-can-aus.html.

If someone knows how to do this, you would be my saviour!

I read through a couple of threads to pull something like this from Adblock Plus (http://dvr-i.rio2016.cbc.ca/dvr/9c8f...nglishnumeris), but I have no clue how to proceed afterwards, so some guidance would be really well appreciated.

ehurdler · #2 08-04-2016, 07:50 AM

I am in the US, so I am not sure if this will be the same for you or not. I have nbcolympics.com, so if you have access to use that instead this will work 100%. The Canada vs. Australia game is there in full replay.

http://stream.nbcolympics.com/womens-soccer-match-2

To capture the video you will need the HDS Link Detector installed in Firefox. Just search for how to do this, install it, configure if you don’t have it already. I don’t remember the process off the top of my head. Make sure you have the updated AdobeHDS.php, set and configured properly too. Again, search for this if you don’t have it already. I am using a Macintosh, not that it makes a difference but telling you my process.

With everything installed and configured, make sure the HDS Link detector add-on is enabled. Now navigate to the game in question. Once the game starts playing you will see the add-on show the banner showing the video fragment link it captures. With this link you can capture the video fragments for this full video, and also the audio fragments for the full audio. The fragments will have to be merged together as well as the video and the audio into one full video. Those processes are rather quick and seamless.

So once the add-on gives you a link, you can modify the link and download the fragments you need and merge them together.

This is an example of the code you will need for this

Code:

Video:

php AdobeHDS.php --quality 7000 --manifest "" --delete


Audio:

php AdobeHDS.php --quality 56 --manifest "" --delete


Join: 

ffmpeg -i video.flv -i audio.flv -c copy joined.mp4

Example, Given this link from the add-on:

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1470317712~exp=1470317832~acl=/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/*~hmac=ee008b15c134623c079477ea82ebbc6bd9ef5a56e3b75d0c134162d2f37f8a30" --delete

With that link, if you take the url between the “”’s and place it within the code strings I posted above it will start capturing the video fragments, then the audio fragments in Terminal (for me since I am using a Macintosh) or a cmd prompt if you are using Windows. The link above will expire in a minute or so. So I always capture the video fragments first. This will download all the video fragments, then it will put them all together with no user interaction into a .flv file. Then I reload the soccer match in the browser to capture a new link in the add-on and paste that within the code string Above. This will as well capture all the audio fragments and put them together. All you will have to do is the last line of code I gave you to Join them with ffmpeg. Will create a video with audio into an .mp4. That will give you the highest video and audio quality possible through the nbcolympics website. You can change the video quality if you would like, but you’ll have to run the -debug script on the add-on link that is captured:

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1470317712~exp=1470317832~acl=/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/*~hmac=ee008b15c134623c079477ea82ebbc6bd9ef5a56e3b75d0c134162d2f37f8a30" -debug

That will give you a whole bunch of info that will just keep scrolling. So let it run and you are looking for the info at the top. So after a few seconds stop the debug script. You want to capture Fragments, not Keyframes.

Code:

Bitrate URL
 7001    QualityLevels(7000000)/Keyframes(video,format=f4m-f4f)
 7000    QualityLevels(7000000)/Fragments(video,format=f4m-f4f)
 4001    QualityLevels(4000000)/Keyframes(video,format=f4m-f4f)
 4000    QualityLevels(4000000)/Fragments(video,format=f4m-f4f)
 2201    QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)
 2200    QualityLevels(2200000)/Fragments(video,format=f4m-f4f)
 1401    QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)
 1400    QualityLevels(1400000)/Fragments(video,format=f4m-f4f)
 901     QualityLevels(900000)/Keyframes(video,format=f4m-f4f)
 900     QualityLevels(900000)/Fragments(video,format=f4m-f4f)
 601     QualityLevels(600000)/Keyframes(video,format=f4m-f4f)
 600     QualityLevels(600000)/Fragments(video,format=f4m-f4f)
 301     QualityLevels(300000)/Keyframes(video,format=f4m-f4f)
 300     QualityLevels(300000)/Fragments(video,format=f4m-f4f)
 56      QualityLevels(56000)/Fragments(audio_en_0,format=f4m-f4f)

Sorry if this is long winded, I just wrote this out quickly with what I do, so if it is too much detail that is why. Any questions let me know. Good luck, maybe this will also work for the CBC site.

mtepper27 · #3 08-04-2016, 04:21 PM

Thank you, that works perfectly!!! I have been searching for help on this for a while.

Would you happen to know how to download WatchESPN streams?

KSV's AdobeHDS Link Detector doesn't detect any links from ESPN streams

Any help would be greatly appreciated.

BusterAvis · #4 08-04-2016, 05:55 PM

Where do you type these 3 codes in?
Command prompt? And if so, where in command prompt?
What folder?

I'm used to using Livestreamer and FFmpeg, both in command prompt.
Never used AdobeHDS before. Thanks.
I'm pretty sure I have some sort of AdobeHDS, because I was able to successfully add the HDS Link Detector "add-on" to Firefox a few minutes ago, and it sniffed out this code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1470355147~exp =1470355267~acl=/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/*~hmac=640871a65a4ccb7fa5f588156d061d7dd2d9f2604b4 dc2b433f73481f32bbb07" --delete

Here are the 3 codes from the first example box above that I'm confused about ............

Video:
php AdobeHDS.php --quality 7000 --manifest "" --delete

Audio:
php AdobeHDS.php --quality 56 --manifest "" --delete

Join:
ffmpeg -i video.flv -i audio.flv -c copy joined.mp4

BusterAvis · #5 08-04-2016, 06:38 PM

I might be on to something.
I found something from a YouTube video:
https://www.youtube.com/watch?v=6DXZHYRqM5A

When trying to grab the manifest code from this NBC Olympics link referenced above:
http://stream.nbcolympics.com/womens-soccer-match-2

I downloaded Grab Any Media for AdobeHDS helper from that YouTube link.
I just don't know the exact code I need to paste into the top left of the image below where you put in the Manifest or Fragment code. Nothing seems to work. I keep trying all the codes I can grab, and I try to click on "Start" in the right (as circled in red) in the image below, but it won't do anything.

Mezar · #6 08-16-2016, 10:32 AM

Quote:

Originally Posted by BusterAvis

I might be on to something.
I found something from a YouTube video:
https://www.youtube.com/watch?v=6DXZHYRqM5A

When trying to grab the manifest code from this NBC Olympics link referenced above:
http://stream.nbcolympics.com/womens-soccer-match-2

I downloaded Grab Any Media for AdobeHDS helper from that YouTube link.

Hallo!
I'm trying to record two streams from Rio Summer Olympics:
http://stream.nbcolympics.com/beach-...quarterfinal-2
&
http://stream.nbcolympics.com/beach-...und-16-match-5

I'm, using Grab Any Media 5 with AdobeHDSHeleper and Hola vpn (I live in Europe).

Unfortunately I have Access Deined!

Can anybody help me?

BusterAvis · #7 08-05-2016, 09:45 PM

Quote:

Originally Posted by mtepper27

Thank you, that works perfectly!!! I have been searching for help on this for a while.

Would you happen to know how to download WatchESPN streams?

KSV's AdobeHDS Link Detector doesn't detect any links from ESPN streams

Any help would be greatly appreciated.

When using Firefox, right after the the "archived" WatchESPN stream begins to play (or during the advertisement, or actually during when you first load up the separate window pop-up screen that shows the WatchESPN event, click these three things on your keyboard at the same time.... Ctrl key + Shift Key + i (i as in iceland)

So once again, it's Ctrl key + Shift Key + i

It'll pop up with the console and you can search for "m3u8"
Once you have the proper m3u8 file (the i.m3u8 file used to be the highest quality in WatchESPN, now it's the j.m3u8 file which sometimes takes longer to pop up).

i.m3u8 used to be 720p, now it's almost 720p (much better than standard still)

j.m3u8 is the true 720p one now.

You can use the m3u8 file in command prompt using FFMPEG.
WatchESPN tends to work better now with FFMPEG than it does with LiveStreamer (it used to work great with LiveStreamer until recently).

raphou · #8 08-06-2016, 01:51 PM

Hello,

I'm also struggling in using AdobeHDS.php with CBC.

I only receive:

Processing manifest info....
Access Denied! Unable to download the manifest.

URL:
http://olympics.cbc.ca/video/vod/soc...d-can-aus.html

After extensive research, I have found the following:

Different URLs for Manifest:
http://dvr-i.rio2016.cbc.ca/dvr/9c8f...##############

http://dvr-i.rio2016.cbc.ca/dvr/9c8f...=m3u8-aapl-v3)

http://dvr-i.rio2016.cbc.ca/dvr/9c8f...nglishnumeris)http://dvr-i.rio2016.cbc.ca/dvr/9c8f...nglishnumeris)

Downloaded Manifest:

First link:

Code:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=300436,RESOLUTION=256x144,CODECS="avc1.42c00c,mp4a.40.2"
QualityLevels(150000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=402636,RESOLUTION=384x216,CODECS="avc1.42c00d,mp4a.40.2"
QualityLevels(250000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=555936,RESOLUTION=384x216,CODECS="avc1.42c00d,mp4a.40.2"
QualityLevels(400000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=760336,RESOLUTION=512x288,CODECS="avc1.42c015,mp4a.40.2"
QualityLevels(600000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=1118036,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2"
QualityLevels(950000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=1577936,RESOLUTION=768x432,CODECS="avc1.4d401e,mp4a.40.2"
QualityLevels(1400000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=2395536,RESOLUTION=1024x576,CODECS="avc1.4d401f,mp4a.40.2"
QualityLevels(2200000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=3673036,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2"
QualityLevels(3450000)/Manifest(video,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-STREAM-INF:BANDWIDTH=138976,CODECS="mp4a.40.2"
QualityLevels(128000)/Manifest(englishnumeris,format=m3u8-aapl-v3)

Second link:

Code:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:6
#EXT-X-PROGRAM-DATE-TIME:1971-08-04T17:18:06.786Z
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501742867866666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501742928026666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501742988186666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501743048346666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501743108506666,format=m3u8-aapl-v3)

(....)

#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501839845786666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501839905946666,format=m3u8-aapl-v3)
#EXTINF:6.016000,no-desc
Fragments(englishnumeris=501839966106666,format=m3u8-aapl-v3)
#EXTINF:2.005333,no-desc
Fragments(englishnumeris=501840026266666,format=m3u8-aapl-v3)
#EXT-X-ENDLIST

Third Link:

Code:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:6
#EXT-X-PROGRAM-DATE-TIME:1971-08-04T17:18:05.680Z
#EXTINF:6.006000,no-desc
Fragments(video=501742856800000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501742916860000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501742976920000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743036980000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743097040000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743157100000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743217160000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743277220000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743337280000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743397340000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743457400000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743517460000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743577520000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743637580000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501743697640000,format=m3u8-aapl-v3,audiotrack=englishnumeris)

(....)

#EXTINF:6.006000,no-desc
Fragments(video=501839913760000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:6.006000,no-desc
Fragments(video=501839973820000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXTINF:2.002000,no-desc
Fragments(video=501840033880000,format=m3u8-aapl-v3,audiotrack=englishnumeris)
#EXT-X-ENDLIST

FROM Mozilla Debug:

Headers:

Request Headers:

Code:

Host: dvr-i.rio2016.cbc.ca
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://olympics.cbc.ca/flash/divaplayer/divaflash/DivaAll.swf?20160806065535
Cookie: s_pers=%######%#####-New%7C1502035270344%3B; AMCV_951720B3535680CB0A490D45%40AdobeOrg=1999109931%7CMCMID%7C88286696642400635392847616661647017606%7CMCAID%7CNONE; hdntl=exp=#######~acl=%2f*~hmac=###########
Connection: keep-alive

Response Headers:

Code:

Access-Control-Allow-Credentials: Server,range,hdntl,hdnts
Access-Control-Allow-Headers: origin,range,hndtl,hdnts
Access-Control-Allow-Methods: GET,POST,OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 86400
Cache-Control: max-age=259200
Content-Length: 171781
Content-Type: application/vnd.apple.mpegurl
Date: Sat, 06 Aug 2016 15:18:04 GMT
Etag: "1505520086413335"
Expires: Tue, 09 Aug 2016 15:18:04 GMT
Pragma: IISMS/6.0,IIS Media Services Premium by Microsoft
Server: Microsoft-IIS/8.5 IISMS/6.0
X-Content-Type-Options: nosniff
X-Powered-By: ASP.NET
x-ms-streaming-duration: 0

What should I do next?

Many thanks

raphou · #9 08-06-2016, 02:14 PM

Also, forgot to mention that HDS Link Detector does not work here, whereas it works for other videos on CBC ( eg: http://www.thecomedynetwork.ca/shows...Package=141171)

Thanks

Impulse1234 · #10 08-04-2016, 07:37 PM

Quote:

Originally Posted by ehurdler

I am in the US, so I am not sure if this will be the same for you or not. I have nbcolympics.com, so if you have access to use that instead this will work 100%. The Canada vs. Australia game is there in full replay.

http://stream.nbcolympics.com/womens-soccer-match-2

To capture the video you will need the HDS Link Detector installed in Firefox. Just search for how to do this, install it, configure if you don’t have it already. I don’t remember the process off the top of my head. Make sure you have the updated AdobeHDS.php, set and configured properly too. Again, search for this if you don’t have it already. I am using a Macintosh, not that it makes a difference but telling you my process.

With everything installed and configured, make sure the HDS Link detector add-on is enabled. Now navigate to the game in question. Once the game starts playing you will see the add-on show the banner showing the video fragment link it captures. With this link you can capture the video fragments for this full video, and also the audio fragments for the full audio. The fragments will have to be merged together as well as the video and the audio into one full video. Those processes are rather quick and seamless.

So once the add-on gives you a link, you can modify the link and download the fragments you need and merge them together.

This is an example of the code you will need for this

Code:

Video:

php AdobeHDS.php --quality 7000 --manifest "" --delete


Audio:

php AdobeHDS.php --quality 56 --manifest "" --delete


Join: 

ffmpeg -i video.flv -i audio.flv -c copy joined.mp4

Example, Given this link from the add-on:

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1470317712~exp=1470317832~acl=/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/*~hmac=ee008b15c134623c079477ea82ebbc6bd9ef5a56e3b75d0c134162d2f37f8a30" --delete

With that link, if you take the url between the “”’s and place it within the code strings I posted above it will start capturing the video fragments, then the audio fragments in Terminal (for me since I am using a Macintosh) or a cmd prompt if you are using Windows. The link above will expire in a minute or so. So I always capture the video fragments first. This will download all the video fragments, then it will put them all together with no user interaction into a .flv file. Then I reload the soccer match in the browser to capture a new link in the add-on and paste that within the code string Above. This will as well capture all the audio fragments and put them together. All you will have to do is the last line of code I gave you to Join them with ffmpeg. Will create a video with audio into an .mp4. That will give you the highest video and audio quality possible through the nbcolympics website. You can change the video quality if you would like, but you’ll have to run the -debug script on the add-on link that is captured:

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1470317712~exp=1470317832~acl=/vod/690c26d3-280d-4d17-9d86-4ef9e406480f/nbc-olympics0803094426.ism/*~hmac=ee008b15c134623c079477ea82ebbc6bd9ef5a56e3b75d0c134162d2f37f8a30" -debug

That will give you a whole bunch of info that will just keep scrolling. So let it run and you are looking for the info at the top. So after a few seconds stop the debug script. You want to capture Fragments, not Keyframes.

Code:

Bitrate URL
 7001    QualityLevels(7000000)/Keyframes(video,format=f4m-f4f)
 7000    QualityLevels(7000000)/Fragments(video,format=f4m-f4f)
 4001    QualityLevels(4000000)/Keyframes(video,format=f4m-f4f)
 4000    QualityLevels(4000000)/Fragments(video,format=f4m-f4f)
 2201    QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)
 2200    QualityLevels(2200000)/Fragments(video,format=f4m-f4f)
 1401    QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)
 1400    QualityLevels(1400000)/Fragments(video,format=f4m-f4f)
 901     QualityLevels(900000)/Keyframes(video,format=f4m-f4f)
 900     QualityLevels(900000)/Fragments(video,format=f4m-f4f)
 601     QualityLevels(600000)/Keyframes(video,format=f4m-f4f)
 600     QualityLevels(600000)/Fragments(video,format=f4m-f4f)
 301     QualityLevels(300000)/Keyframes(video,format=f4m-f4f)
 300     QualityLevels(300000)/Fragments(video,format=f4m-f4f)
 56      QualityLevels(56000)/Fragments(audio_en_0,format=f4m-f4f)

Sorry if this is long winded, I just wrote this out quickly with what I do, so if it is too much detail that is why. Any questions let me know. Good luck, maybe this will also work for the CBC site.

Turns out I can't access NBC Olympics, region-locked of course :/
And I tried downloading HDS Link Detector (the addon for Firefox), and when I tried playing the video nothing popped up in terms of coding. :/