Page 2 of 2

Show 40 post(s) from this thread on one page

Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)

- Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)

- -

Stream capture from NBCOlympics

(http://stream-recorder.com/forum/showthread.php?t=16887)

nwtim

02-24-2014 04:01 PM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by KSV (Post 64938)

I have looked into the issue and pushed an update to fix it...

PS: regarding the problem with converting some of the fragments downloded with DTA it's actually DTA's fault. sometimes it saves the error message sent by server instead of actual fragment data. also you don't have to rename the downloaded fragments. read the wiki again to understand how to do it properly.

That's an awesome fix! Thanks!!

As for the renaming, the reason I was doing it was because even when I tried to use it with a base filename it seemed to require that the first fragment was "1", while the files I was working with at the time had fragments starting at about Seg1-Frag620 or so, and the base filename method didn't seem to like that.

nwtim

02-24-2014 11:28 PM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by KSV (Post 64938)

...
When you run the updated script with debug switch it will show the following content.

Code:

php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -debug

...

This is working great with the special feature videos, but I was also hoping to download a number of event replays which have a manifest that includes a token, and I can't seem to get anything with those.

Here's an example of an event replay's manifest script as captured with the HDS Links Detector add-on for Firefox:

Code:

"http://olystreameast.nbcolympics.com/vod/a3ceaf1c-15b3-4cb0-a83a-00af8e5e08de/pairs-short-program0211061442.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1393308775~exp=1393308895~acl=/vod/a3ceaf1c-15b3-4cb0-a83a-00af8e5e08de/pairs-short-program0211061442.ism/*~hmac=2defc70d894a7774c2fbf749cdc5a3458c9151f212b608410b10a44df134921e"

Can you show me what to parse out in order for the -debug switch to work with such a manifest?

Thanks!

KSV	02-25-2014 07:34 AM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by nwtim (Post 64949)

As for the renaming, the reason I was doing it was because even when I tried to use it with a base filename it seemed to require that the first fragment was "1", while the files I was working with at the time had fragments starting at about Seg1-Frag620 or so, and the base filename method didn't seem to like that.

You can use start switch to specify the starting fragment.

Code:

php AdobeHDS.php --fragments "Fragments(video,format=f4m-f4f)Seg1-Frag" --start 620

Code:

php AdobeHDS.php "Fragments(video,format=f4m-f4f)Seg1-Frag" --start 620

Quote:

Originally Posted by nwtim (Post 64953)

Code:

"http://olystreameast.nbcolympics.com/vod/a3ceaf1c-15b3-4cb0-a83a-00af8e5e08de/pairs-short-program0211061442.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1393308775~exp=1393308895~acl=/vod/a3ceaf1c-15b3-4cb0-a83a-00af8e5e08de/pairs-short-program0211061442.ism/*~hmac=2defc70d894a7774c2fbf749cdc5a3458c9151f212b608410b10a44df134921e"

Can you show me what to parse out in order for the -debug switch to work with such a manifest?

i am not sure what you are asking for. you can append "--debug" switch to the command line provided by "HDS Link Detector" addon without modifying the command line in any other way.

blimey

02-25-2014 08:23 AM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by nwtim (Post 64953)

This is working great with the special feature videos, but I was also hoping to download a number of event replays which have a manifest that includes a token, and I can't seem to get anything with those.
.
.
Can you show me what to parse out in order for the -debug switch to work with such a manifest?

Thanks!

The event programs use authenticated time-limited urls for the f4m manifest. The syntax would be much like the example given by ehurdler in post #10 of this thread... only adjusted to reflect your authentication particulars.

You probably don't want to waste time, or an authenticated f4m manifest url access, doing a debug command. You know the quality levels already, have separate command windows open and do one each for the audio and video before the manifest url expires. Or you could do them consecutively, but you'd have to get a new manifest url before you do the second one as the first will likely have expired.

What was the problem you had using the ios url (m3u8) w/ffmpeg (as provided by chap) for an event program? I had no trouble, although I did the audio and video in separate consecutive ffmpeg sequences, rather than as two inputs to ffmpeg as a single command as chap has it. The ios urls for the manifest need no authentication even for event programs.

nwtim

02-25-2014 12:01 PM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by KSV (Post 64958)

You can use start switch to specify the starting fragment.

Code:

php AdobeHDS.php --fragments "Fragments(video,format=f4m-f4f)Seg1-Frag" --start 620

Code:

php AdobeHDS.php "Fragments(video,format=f4m-f4f)Seg1-Frag" --start 620

Oops - my bad. Being new to the whole "Fragment" world, I guess I wasn't confident as to the exact syntax for the start switch so I was taking the extra minute to rename the files. I won't waste my time on that anymore. Thanks!

Quote:

i am not sure what you are asking for. you can append "--debug" switch to the command line provided by "HDS Link Detector" addon without modifying the command line in any other way.

Wouldn't you know it - I couldn't get it to work last night on my other PC and just now when I tried it it worked. Must be that my other PC is clogged up with Windows Media Center, recording multiple shows at once off our HDHomeRun Tuners, and is only Win7x86. With that PC I was getting no quality header from the manifest or an "Unauthorized" error when omitting the "?token=" stuff from the HDS Link.

Thanks for all your help!

nwtim

02-25-2014 12:26 PM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by blimey (Post 64959)

Right - I normally don't take the time to use the debug switch. I was only using it looking for possible explanations why I couldn't get a couple event replays to work last night with KSV's Adobe HDS downloader.

After giving up on that approach, I tried chap's ffmpeg method of getting the m3u8 streams and had amazing success grabbing a couple 4-hour events. Perhaps it was due to being able to bypass whatever issue I was having with the tokens I was getting (I never waited more than a minute between getting the authentication token and trying to run with it - it just wasn't working).

I'm thinking the problems I've had with these downloads could be - and probably are - tied to having spotty internet service both at home and at work over the last week or so (using Comcast in greater Seattle area). I'm not sure how tolerant these methods are of connection interruptions, but last night and this morning seem to have been far better in that regard, and correspondingly any download attempts I've made have been successful.

For example, one I had great difficulty with a couple days ago - the Women's Moguls Qualification 1 - now seems to be coming down without a hitch using KSV's Adobe HDS downloader. Since the connection is throttled down a bit, I imagine I'll need to get a new authentication token for the audio, but so far so good.

Thanks!

nwtim

02-25-2014 04:28 PM

Re: Stream capture from NBCOlympics

Okay, I've got a problem like the one I was getting last night. It's for the "Figure Skating: Team Event: Men's SP, Pairs SP". The HDS Link I get is

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/724e2f71-42bf-43c2-a878-ff6c305e318b/team--men-s-sp--pairs-sp0206063549.ism/manifest(format=f4m-f4f).f4m?token=st=1393370184~exp=1393370304~acl=/vod/724e2f71-42bf-43c2-a878-ff6c305e318b/team--men-s-sp--pairs-sp0206063549.ism/*~hmac=f96ea87266758164ee32a2a220d6f3f6ac3773ee108bdd6da1279c86199f9c88" --delete

and when I replace --delete with --debug I get "Access Denied! Unable to download the manifest." And this is while I have it playing in the browser and keep refreshing the HDS link.

I tried chap's ffmpeg method to get the m3u8 streams and it isn't working either using the string

Code:

http://olystreameast.nbcolympics.com/vod/724e2f71-42bf-43c2-a878-ff6c305e318b/team--men-s-sp--pairs-sp0206063549.ism

in the syntax.

chap	02-25-2014 05:07 PM

Re: Stream capture from NBCOlympics

nwtim
try

Code:

ffmpeg -i "http://olystreameast.nbcolympics.com/vod/724e2f71-42bf-43c2-a878-ff6c305e318b/team--men-s-sp--pairs-sp0206063549.ism/QualityLevels(56000)/Manifest(audio_en_0,format=m3u8-aapl)" -i "http://olystreameast.nbcolympics.com/vod/724e2f71-42bf-43c2-a878-ff6c305e318b/team--men-s-sp--pairs-sp0206063549.ism/QualityLevels(3450000)/Manifest(video,format=m3u8-aapl)" -c copy t.ts

nwtim

02-25-2014 05:16 PM

Re: Stream capture from NBCOlympics

Somehow the Manifest argument I had in my template had been changed to "(audio_en_0,format=m3u8-aapl,filtername=vodcut)" and the same ",filtername=vodcut" for the video. Now it's running just fine.

Thanks chap!

blimey

02-26-2014 12:58 AM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by nwtim (Post 64967)

I get "Access Denied! Unable to download the manifest." And this is while I have it playing in the browser and keep refreshing the HDS link

I've now seen a few of these "Access Denied" errors for some authentication-required events. The f4m manifest link, or even a f4f fragment link, reports this error if used outside of the browser. But if the link is accessed inside the browser (in which the authentication is stored), it is okay. There must be browser cookies, or more likely flash/adobe cookies. Some special files are downloaded from adobe the first time authentication goes through.

nwtim

02-26-2014 12:14 PM

Re: Stream capture from NBCOlympics

I don't see those errors when I'm using chap's ffmpeg method of getting the m3u8 streams, but there are several that I cannot get the entire event of and I can't seem to use the -ss and -t switches to steer around problem areas with his method. In those cases where I can't use the ffmpeg method and the AdobeHDS.php method returns the unauthorized access error, I don't know of any other solution than to try to do the best I can with DownloadThemAll.

ehurdler

02-26-2014 02:00 PM

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by nwtim (Post 64979)

I have downloaded every event I have wanted so far with no errors at all with AdobeHDS.php since KSV updated the script. Can you give me an example of an event replay that is not working for you? As well and explaining your process?

I have a folder on my desktop with the AdobeHDS.php script in it and nothing else. I have a text document opened at all times with only the template of:

Code:

php AdobeHDS.php --manifest "" -quality 56 --delete



php AdobeHDS.php --manifest "" -quality 3450 --delete

I open for example:

HTML Code:

http://stream.nbcolympics.com/cross-country/winter/14193/

I quickly open Tool->Web Developer->Web Console. I only have the "Net" tab opened. As the event reply is starting, I will click on the first line I see that has ".ism/manifest....etc.etc. in it to open the "Inspect Network Request" window. Copy the full "Request URL:" ASAP. That that and paste it in between the " "s on my text document template for the audio first.

Example:

Code:

php AdobeHDS.php --manifest "http://olystreameast.nbcolympics.com/vod/a9b40645-ee55-47e0-8e6c-fe848ead2493/cc--4x5km-relay0215001906.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1393447534~exp=1393447654~acl=/vod/a9b40645-ee55-47e0-8e6c-fe848ead2493/cc--4x5km-relay0215001906.ism/*~hmac=a2c9229ca75d27d306db55cfe9376ce0af3e961c99b0eed6eef6f8cb8ecfe340" -quality 56 --delete

I immediately run this command in Terminal (as I am using a Macintosh). Run the command ASAP as it does have an expiration as you know, this way I guess it won't time out, give errors, access denied, etc. I start to see the captured fragments number go up, up, etc. I just pause the event replay, and keep the website open. (not sure if this makes a difference, but it is not open long, plus I will refresh it to capture video segments in a few minutes.)

I let that capture all the fragments into the specified folder that I am running AdobeHDS.php to/from. Once that is done, I rename the .flv file that was created to something like "4x5kmRelayAudio.flv" I manually delete all the fragments now, as for some reason the --delete code doesn't work, but I don't care I just manually delete them.

Now refresh the webpage, so to grab the ".ism/manifest...etc...etc." url again just as done for the audio portion above. Copy url into my template, paste whole command into Terminal again, let it capture all the fragments. Keep page open, pause video, wait till complete. Once it captures them all I rename the .flv file again to "4x5kmRelayVideo.flv". Now I use an app to change audio to aac format and the video to .mkv format. I use another app to mux the track together. I have gotten a flawless video everytime this way. Using FFMPEG I kept getting audio issues. I couldn't edit the resulting videos at all. This way I am able to edit out portions with no issue. Look great on USB plugged to Flatscreen, or streaming to/through AppleTV.

Sorry if I went into too much detail. It took me a long time to get MY process down based on KSV, blimey, and chaps posts to get it working perfectly for me! Thanks everyone BTW! NWTIM, if you have any other issues let us know.

nwtim

02-26-2014 05:33 PM

Re: Stream capture from NBCOlympics

Thanks for all that detail, ehurdler. I went over it carefully to see if there was anything you were doing that was fundamentally different than what I'm doing. The biggest difference may be that I'm getting the manifest file from KSV's HDS Link Detector Add-on or filtering the Web Console feedback for .f4m or Seg (when specific frag numbered files are of interest to me) files.

In your experience, just how quickly do these authentication tokens typically expire? I run everything from batch files, but usually never have a delay of more than about 30 seconds from the time I paste the text string in to the time I run it.

I don't have time at the moment, but tonight I'll take another stab at one of the videos I've had trouble with (they're all event replays, of which I'm having about a 60-70% success rate with).

nwtim

02-26-2014 11:06 PM

Re: Stream capture from NBCOlympics

I've solved my problem - all having to do with using expired authentication tokens. I kept re-grabbing the HDS Link Detector script to plug it into my batch file and at some point I "detected" that it hadn't changed from the last time I grabbed it, so I made the mistaken assumption that it just wasn't expiring that fast.

For several days it didn't occur to me that the HDS Link Detector might not be getting its last value flushed as long as the browser was open and/or the cache not cleared. It may be even simpler than this, but ever since I began clearing the browser's cache and closing it and then re-opening it to re-grab the manifest script from HDS Link Detector, the AdobeHDS.php method has been working flawlessly - even on the videos I was having the toughest time with.

I was/am using a batch file with just these lines in it:

Code:

"" -quality 56 -outfile Audio.flv



pause



"" -quality 3450 -outfile Video.flv



pause



ffmpeg -i Audio.flv -i Video.flv -c copy FINAL.mp4



pause

The double quotes get replaced with the script that populates the clipboard when I click on the HDS Link Detector pop-up. The pauses are so I can monitor any feedback before continuing on to the next step. If something went wrong I can simply abort the batch file (Ctrl-C). I also noticed that the ffmpeg mux didn't work properly unless the pause was there and I waited a couple seconds to continue. BTW, the HDS Link Detector pop-up script includes the --delete switch and it seems to actually delete all the fragments when I run the full batch file but not when I run only one part of it (remark out the other operations).

Quote:

Originally Posted by ehurdler (Post 64981)

...
Now refresh the webpage, so to grab the ".ism/manifest...etc...etc." url again just as done for the audio portion above.

This is where I think I was having my problem. I WAS refreshing the page on a consistent basis (many times, actually), but the script that HDS Link Detector was putting on my clipboard was NOT changing - which is probably why I was lulled into thinking the expired tokens were still current.

Quote:

...Now I use an app to change audio to aac format and the video to .mkv format. I use another app to mux the track together. I have gotten a flawless video everytime this way. Using FFMPEG I kept getting audio issues. I couldn't edit the resulting videos at all. This way I am able to edit out portions with no issue. Look great on USB plugged to Flatscreen, or streaming to/through AppleTV...

I'm not getting audio issues with ffmpeg at all as long as I'm using AdobeHDS.php to merge the fragment files. I do note however, that sometimes the number of audio fragments has been one more than the number of video fragments, which may or may not be related to my finding that my audio sync was normally spot on with DownloadThemAll fragments when I forced "audio...Seg1-Frag[x]" to align with "video...Seg1-Frag[x+1]".

Anyway, all is good now with AdobeHDS.php doing the downloading and joining and ffmpeg doing the muxing and converting to MP4.

Thanks for everyone's contributions here.

james667

03-17-2014 10:51 AM

Re: Stream capture from NBCOlympics

Has anyone had success downloading any Olympic streams from cbc.ca? I've been at this for a while. I want this "http://olympics.cbc.ca/videos/live/video/speed-skating-women-1500m-finals-webcast.html" I have tried Orbit downloader and ended up with a lot of unplayable mov files? I'm using Ismdownloader right now and need some help. I believe I filtered out the ism/manifest link with firefox console, now how would I input that into ismdownloader? When I open the ismdownloader.exe it closes right away. I'm missing a few steps. If anyone can help or has a better way to grab from cbc please let me know.
Thanks in advance.

x-9er

04-18-2014 12:39 AM

Re: Stream capture from NBCOlympics

Has anyone figured out a way to capture the closed captioning from these broadcasts?

All times are GMT -6. The time now is 12:42 PM.

Page 2 of 2

Show 40 post(s) from this thread on one page