Stream capture from NBCOlympics

[nwtim]

I don't see those errors when I'm using chap's ffmpeg method of getting the m3u8 streams, but there are several that I cannot get the entire event of and I can't seem to use the -ss and -t switches to steer around problem areas with his method. In those cases where I can't use the ffmpeg method and the AdobeHDS.php method returns the unauthorized access error, I don't know of any other solution than to try to do the best I can with DownloadThemAll.

[ehurdler]

Originally Posted by nwtim

I don't see those errors when I'm using chap's ffmpeg method of getting the m3u8 streams, but there are several that I cannot get the entire event of and I can't seem to use the -ss and -t switches to steer around problem areas with his method. In those cases where I can't use the ffmpeg method and the AdobeHDS.php method returns the unauthorized access error, I don't know of any other solution than to try to do the best I can with DownloadThemAll.

I have downloaded every event I have wanted so far with no errors at all with AdobeHDS.php since KSV updated the script. Can you give me an example of an event replay that is not working for you? As well and explaining your process?

I have a folder on my desktop with the AdobeHDS.php script in it and nothing else. I have a text document opened at all times with only the template of:

php AdobeHDS.php --manifest "" -quality 56 --delete

php AdobeHDS.php --manifest "" -quality 3450 --delete

I open for example:

http://stream.nbcolympics.com/cross-country/winter/14193/

I quickly open Tool->Web Developer->Web Console. I only have the "Net" tab opened. As the event reply is starting, I will click on the first line I see that has ".ism/manifest....etc.etc. in it to open the "Inspect Network Request" window. Copy the full "Request URL:" ASAP. That that and paste it in between the " "s on my text document template for the audio first.

Example:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/a9b40645-ee55-47e0-8e6c-fe848ead2493/cc--4x5km-relay0215001906.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1393447534~exp=1393447654~acl=/vod/a9b40645-ee55-47e0-8e6c-fe848ead2493/cc--4x5km-relay0215001906.ism/*~hmac=a2c9229ca75d27d306db55cfe9376ce0af3e961c99b0eed6eef6f8cb8ecfe340" -quality 56 --delete

I immediately run this command in Terminal (as I am using a Macintosh). Run the command ASAP as it does have an expiration as you know, this way I guess it won't time out, give errors, access denied, etc. I start to see the captured fragments number go up, up, etc. I just pause the event replay, and keep the website open. (not sure if this makes a difference, but it is not open long, plus I will refresh it to capture video segments in a few minutes.)

I let that capture all the fragments into the specified folder that I am running AdobeHDS.php to/from. Once that is done, I rename the .flv file that was created to something like "4x5kmRelayAudio.flv" I manually delete all the fragments now, as for some reason the --delete code doesn't work, but I don't care I just manually delete them.

Now refresh the webpage, so to grab the ".ism/manifest...etc...etc." url again just as done for the audio portion above. Copy url into my template, paste whole command into Terminal again, let it capture all the fragments. Keep page open, pause video, wait till complete. Once it captures them all I rename the .flv file again to "4x5kmRelayVideo.flv". Now I use an app to change audio to aac format and the video to .mkv format. I use another app to mux the track together. I have gotten a flawless video everytime this way. Using FFMPEG I kept getting audio issues. I couldn't edit the resulting videos at all. This way I am able to edit out portions with no issue. Look great on USB plugged to Flatscreen, or streaming to/through AppleTV.

Sorry if I went into too much detail. It took me a long time to get MY process down based on KSV, blimey, and chaps posts to get it working perfectly for me! Thanks everyone BTW! NWTIM, if you have any other issues let us know.

[nwtim]

Thanks for all that detail, ehurdler. I went over it carefully to see if there was anything you were doing that was fundamentally different than what I'm doing. The biggest difference may be that I'm getting the manifest file from KSV's HDS Link Detector Add-on or filtering the Web Console feedback for .f4m or Seg (when specific frag numbered files are of interest to me) files.

In your experience, just how quickly do these authentication tokens typically expire? I run everything from batch files, but usually never have a delay of more than about 30 seconds from the time I paste the text string in to the time I run it.

I don't have time at the moment, but tonight I'll take another stab at one of the videos I've had trouble with (they're all event replays, of which I'm having about a 60-70% success rate with).

[nwtim]

I've solved my problem - all having to do with using expired authentication tokens. I kept re-grabbing the HDS Link Detector script to plug it into my batch file and at some point I "detected" that it hadn't changed from the last time I grabbed it, so I made the mistaken assumption that it just wasn't expiring that fast.

For several days it didn't occur to me that the HDS Link Detector might not be getting its last value flushed as long as the browser was open and/or the cache not cleared. It may be even simpler than this, but ever since I began clearing the browser's cache and closing it and then re-opening it to re-grab the manifest script from HDS Link Detector, the AdobeHDS.php method has been working flawlessly - even on the videos I was having the toughest time with.

I was/am using a batch file with just these lines in it:

"" -quality 56 -outfile Audio.flv

pause

"" -quality 3450 -outfile Video.flv

pause

ffmpeg -i Audio.flv -i Video.flv -c copy FINAL.mp4

pause

The double quotes get replaced with the script that populates the clipboard when I click on the HDS Link Detector pop-up. The pauses are so I can monitor any feedback before continuing on to the next step. If something went wrong I can simply abort the batch file (Ctrl-C). I also noticed that the ffmpeg mux didn't work properly unless the pause was there and I waited a couple seconds to continue. BTW, the HDS Link Detector pop-up script includes the --delete switch and it seems to actually delete all the fragments when I run the full batch file but not when I run only one part of it (remark out the other operations).

Originally Posted by ehurdler

...
Now refresh the webpage, so to grab the ".ism/manifest...etc...etc." url again just as done for the audio portion above.

This is where I think I was having my problem. I WAS refreshing the page on a consistent basis (many times, actually), but the script that HDS Link Detector was putting on my clipboard was NOT changing - which is probably why I was lulled into thinking the expired tokens were still current.

...Now I use an app to change audio to aac format and the video to .mkv format. I use another app to mux the track together. I have gotten a flawless video everytime this way. Using FFMPEG I kept getting audio issues. I couldn't edit the resulting videos at all. This way I am able to edit out portions with no issue. Look great on USB plugged to Flatscreen, or streaming to/through AppleTV...

I'm not getting audio issues with ffmpeg at all as long as I'm using AdobeHDS.php to merge the fragment files. I do note however, that sometimes the number of audio fragments has been one more than the number of video fragments, which may or may not be related to my finding that my audio sync was normally spot on with DownloadThemAll fragments when I forced "audio...Seg1-Frag[x]" to align with "video...Seg1-Frag[x+1]".

Anyway, all is good now with AdobeHDS.php doing the downloading and joining and ffmpeg doing the muxing and converting to MP4.

Thanks for everyone's contributions here.

[james667]

Has anyone had success downloading any Olympic streams from cbc.ca? I've been at this for a while. I want this "http://olympics.cbc.ca/videos/live/video/speed-skating-women-1500m-finals-webcast.html" I have tried Orbit downloader and ended up with a lot of unplayable mov files? I'm using Ismdownloader right now and need some help. I believe I filtered out the ism/manifest link with firefox console, now how would I input that into ismdownloader? When I open the ismdownloader.exe it closes right away. I'm missing a few steps. If anyone can help or has a better way to grab from cbc please let me know.
Thanks in advance.

[x-9er]

Has anyone figured out a way to capture the closed captioning from these broadcasts?