Stream capture from NBCOlympics

[nwtim]

You got a lot further than I did. I haven't even been able to identify the proper manifest URL. I'm using an older version of Firefox (17.0.5) for an add-on (CacheViewer Fx21) I frequently use that doesn't work in newer versions, so I can't use KSV's HDS Link Detector add-on and I can't seem to parse it out using Web Console or Coojah.

I understand what you're describing regarding the improper playback and audio problems. I've had this happen when audio and video streams don't mux properly, so it must have something to do with the way the fragments are being joined together. Have you tried grabbing a lower quality stream to see if the problem is there as well?

[ehurdler]

NWTIM:
As for your version of Firefox I am unsure any difference than the one I am using. But just open up Tools->Web Developer->Web Console.

Once Web Console is open, on the tabs across the top of the console turn everything off but Net. In the filter/search box on the right of the console search for Manifest. I usually open the console on the page I am viewing, do the search, then refresh the page just to make sure I see everything I am searching for the start of the page loading. Give that a try, try my method I described earlier, see if you get same result. I am trying to capture the USA vs CZE Hockey game from today.

[nwtim]

I got a good manifest link for an NBCOlympics video, but when I try to run it (I basically just copied it into your string and added a --duration 60 switch) I get a "Could not open input file: AdobeHDS.php" error. I'm wondering if I'm just missing an --auth switch string or something.

Do you know anything about that?

[ehurdler]

Can I see your code you entered in?

[nwtim]

Originally Posted by ehurdler

Can I see your code you entered in?

Sure, I just started one of the most recent files on the NBCOlympics|Videos page and parsed out the manifest link.

php AdobeHDS.php --quality high --duration 60 --manifest "http://olystreameast.nbcolympics.com/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/manifest(format=f4m-f4f,filtername=vodcut).f4m?token=st=1392875416~exp=1392875536~acl=/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/*~hmac=89fab799b8075f745646451f1c89617a1c9045c3fa96923a90cff00dd02ca13d" --delete

It's probably expired by now, or perhaps it's IP address dependent.

[ehurdler]

I was able to get downthemall to download the fragments. Or at least I think I got everything. I got a bunch of files what I believe should be audio, all in sequence named "Fragments(audio_en_0,format=f4m-f4f)Seg1-Frag735", etc, etc.

Also got what I would think is video, all in sequence named "Fragments(video,format=f4m-f4f)Seg1-Frag1", etc, etc.

What I guess I figured out, I had been only downloading the keyframe files ONLY when using the AdobeHDS.php script. I dont know what to do with those keyframe files. It does stitch them together, and I get what I described as a video that looks like it is being fast forwarded. Any idea what to do with the keyframe, audio fragments, and video fragments???

[nwtim]

Originally Posted by ehurdler

I was able to get downthemall to download the fragments. Or at least I think I got everything. I got a bunch of files what I believe should be audio, all in sequence named "Fragments(audio_en_0,format=f4m-f4f)Seg1-Frag735", etc, etc.

Also got what I would think is video, all in sequence named "Fragments(video,format=f4m-f4f)Seg1-Frag1", etc, etc.

What I guess I figured out, I had been only downloading the keyframe files ONLY when using the AdobeHDS.php script. I dont know what to do with those keyframe files. It does stitch them together, and I get what I described as a video that looks like it is being fast forwarded. Any idea what to do with the keyframe, audio fragments, and video fragments???

This is starting to add up to something, but I don't know the solutions yet.

I recall that a few months back YouTube made a change to their video delivery that caused most downloaders to not be able to get the 1080p or 4K video without a muxing operation. Several of them began to charge money for the 1080p and 4K streams because they had to do the muxing on their own servers. JDownloader did it by using the client's PC, but requiring ffmpeg on the PC.

Perhaps what we have to do with these audio and video stream fragments is very similar to what the downloaders are doing with YouTube's highest quality videos. I just came across KSV's AdobeHDS.php yesterday, so I don't know if it can do this kind of merging or not or if this is just some kind of DRM issue. I'll try to get some time later to poke around it a bit.

[nwtim]

Okay, I see what you're talking about now because I went ahead and updated my Firefox so I could use the HDS Links Detector add-on. I just finished downloading the Women's Gold Medal Hockey game and I had it keep all the fragments. The first thing I noticed was that these fragments were really very small files - in the neighborhood of 100 KB each - which is about what you would expect from a snapshot you might use in creating stop-motion video. There was no audio at all.

In contrast, when I used DownloadThemAll, I was getting fragments averaging more than 2-3 MB each.

So, using DownloadThemAll, I downloaded about 40 fragments and renamed them to 1.f4f, 2.f4f, 3.f4f etc. Then I joined them by just running "php AdobeHDS.php" in a command window of that folder. The resulting video (Joined.flv) plays and looks fantastic, but there's just no sound.

Please tell me how you got the audio fragments, because I somehow missed those.

Anyway, I'm kinda thinking we might be able to join the audio files the same way I joined the video files using AdobeHDS and then once we have those we should be able to mux them together.

[ehurdler]

Can you please give me the code, syntax you used to join the video files? Also, what were the resulting files you were able to download with downthemall called? As well as how did you download them exactly? I did what you said in the post from last night and all the fragments/files are "Fragments(video,format=f4m-f4f)Seg1-Frag1" "Fragments(video,format=f4m-f4f)Seg1-Frag2", etc, etc. Those are the names of the files, no ext at all. I have tried adding .f4f and attempting to join together and I am getting nowhere.

[nwtim]

It looks like you got exactly the same kind of files I got, and I believe we're both using the same method. Mine were named just like yours, ending in "Seg1-Frag[sequence number]" with no extension.

Based on KSV's wiki link (https://github.com/K-S-V/Scripts/wiki) under "Offline fragment processing", I decided to rename them all to the form 1.f4f, 2.f4f, 3.f4f, and so on (I use Flexible Renamer to make it a very quick and simple operation), but the wiki states that the extension is optional.

I opened up a command window in the folder with all the fragment files and simply typed "php AdobeHDS.php". That's it - the resulting file, named Joined.flv, was created almost instantaneously even though the 20-minute video was over 500 MB in size.

Now we just have to figure out how to get it with audio...

Edit:
Okay, I've got it. I'm in the process of downloading 2000 audio and 2000 video fragments right now so I can play with portions of them. The first thing I'm going to try is join some audio files with AdobeHDS.