Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)

- Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)

- -

Stream capture from NBCOlympics

(http://stream-recorder.com/forum/showthread.php?t=16887)

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by chap (Post 64876)

try simple

Code:

ffmpeg -i "http://olystreameast.nbcolympics.com/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/QualityLevels(56000)/Manifest(audio_en_0,format=m3u8-aapl,filtername=vodcut)" -i "http://olystreameast.nbcolympics.com/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/QualityLevels(3450000)/Manifest(video,format=m3u8-aapl,filtername=vodcut)" -c copy t.ts

chap, would you be able to explain how you got the syntax for this? I am trying to capture some of the non-sports videos and the syntax you provided does not work for those. Example, this link: http://www.nbcolympics.com/video/exa...act-space-race

If I navigate to what comes up to be the manifest file, I can see all the options for video stream index, and audio stream index. I just can't get the proper code/syntax to reproduce anything that will stream download with ffmpeg. Quick copy/paste from manifest is:

Code:

<SmoothStreamingMedia MajorVersion="2" MinorVersion="0" Duration="11395717666">

  <StreamIndex Type="video" Chunks="570" QualityLevels="7" MaxWidth="1280" MaxHeight="720" DisplayWidth="1280" DisplayHeight="720" Url="QualityLevels({bitrate})/Fragments(video={start time})">

    <QualityLevel Index="0" Bitrate="3450000" FourCC="H264" MaxWidth="1280" MaxHeight="720" CodecPrivateData="0000000127640029AC723014016EC04400000FA40003A983888001A5240001A524EF7B80F844229C0000000128FFBCB0" NominalBitrate="4485000"/>

    <QualityLevel Index="1" Bitrate="2200000" FourCC="H264" MaxWidth="960" MaxHeight="540" CodecPrivateData="00000001274D401FB9181E022FDE022000007D20001D4C1C0C000432380008647DEF701F088453800000000128FFBC80" NominalBitrate="2860000"/>

    <QualityLevel Index="2" Bitrate="1400000" FourCC="H264" MaxWidth="704" MaxHeight="396" CodecPrivateData="00000001274D401FB91816067F7808800001F4800075307030002AB9800155CF7BDC07C22114E00000000128FFBC80" NominalBitrate="1820000"/>

    <QualityLevel Index="3" Bitrate="900000" FourCC="H264" MaxWidth="640" MaxHeight="360" CodecPrivateData="000000012742C01EB9101405FF2E022000007D20001D4C1C480036EE0006DDDDEF701F08846A0000000128DE3C80" NominalBitrate="1170000"/>

    <QualityLevel Index="4" Bitrate="600000" FourCC="H264" MaxWidth="512" MaxHeight="288" CodecPrivateData="000000016742C01EB6808025B01100000303E90000EA60E06000927C00124FE6A200F8B1750000000168CE3C80" NominalBitrate="780000"/>

    <QualityLevel Index="5" Bitrate="400000" FourCC="H264" MaxWidth="340" MaxHeight="192" CodecPrivateData="000000016742C01EB681619E7F011000003E90000EA60E280030D40061AE6A200F8B17500000000168CE3C80" NominalBitrate="520000"/>

    <QualityLevel Index="6" Bitrate="200000" FourCC="H264" MaxWidth="340" MaxHeight="192" CodecPrivateData="000000016742C01EB681619E7F011000003E90000EA60E060030D40061AE6A200F8B17500000000168CE3C80" NominalBitrate="260000"/>

    <c n="0" d="20020000"/>

...

...

...

    <c n="568" d="20020000"/>

    <c n="569" d="4337666"/>

</StreamIndex>

  <StreamIndex Type="audio" Name="audio_1" Chunks="570" QualityLevels="1" Language="eng" Url="QualityLevels({bitrate})/Fragments(audio_1={start time})">

    <QualityLevel Index="0" Bitrate="48000" FourCC="AACH" SamplingRate="22050" Channels="2" BitsPerSample="16" PacketSize="4" AudioTag="255" CodecPrivateData="139056E5A0"/>

    <c n="0" d="21362358"/>

...

...

...

    <c n="568" d="19969161"/>

    <c n="569" d="2786395"/>

  </StreamIndex>

</SmoothStreamingMedia>

Any help would be greatly appreciated. Thank you.

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by ehurdler (Post 64912)

chap, would you be able to explain how you got the syntax for this? I am trying to capture some of the non-sports videos and the syntax you provided does not work for those. Example, this link:

Code:

h..p://www.nbcolympics.com/video/examining-impact-space-race

The fragment urls still work just fine. It is almost as easy to download all the fragments with a batch downloader and then use the script to join the fragments, and ffmpeg to mux. The sync came out fine here.

For example, for the the link you reference above;

Code:

video:



h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(3450000)/Fragments(video,format=f4m-f4f)Seg1-Frag1

.

.

.

h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(3450000)/Fragments(video,format=f4m-f4f)Seg1-Frag569



audio:



h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)Seg1-Frag1

.

.

.

h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)Seg1-Frag569

After running the script seperately on each set of fragments, mux the audio and video with ffmpeg:

Code:

ffmpeg -i Joinedvideo.flv -i Joinedaudio.flv -c copy video_audio.mp4

Using IDM batch download for the fragments seems faster than ffmpeg. Also note, for videos that don't require any provider authentication, like the one you reference here, TD will download and join the fragments for both the video and the audio (all quality levels offered), you just have to mux them.

Re: Stream capture from NBCOlympics

one word: TUBEDIGGER

Re: Stream capture from NBCOlympics

TD is great, but it seems to only offer a full manifest (with all the quality selections) for public videos.

For videos requiring provider authentication, it would only download one video quality (not the highest), and didn't offer to download audio at all. If you had a different experience with non-public (authentication required) videos using TD, please share.

Re: Stream capture from NBCOlympics

Thank you Blimey!

I had downloaded the individual fragments originally with DownThemAll for that video link I had posted, Space Race video.... Well that had worked, but seemed to be too long of a process. I had seen what chap wrote a few posts before, and that was working for capturing the event/sports. As I stated it did not work for capturing the features/news links off the site.

I had done some more searching after your most recent post. I installed wget, made a temp url.txt list. I keep quickly can now copy/paste the correct url in the list, add or remove lines to count for more or less fragment files. I can quickly run wget with the url list for audio and video fragments, very quick to download them into separate folders. I have to run a renaming app to rename to 1.f4f, 2.f4f, etc. etc. I am on a Macintosh so I use A Better File Rename 8. I am looking into how to use wget to download fragments and rename all in the same process.

Run

Code:

php AdobeHDS.php

for each type, audio/video. Gives me the Joined.flv for both, rename to JoinedAudio.flv, JoinedVideo.flv. Quickly run with ffmpeg as you stated

Code:

ffmpeg -i Joinedvideo.flv -i Joinedaudio.flv -c copy video_audio.mp4

I am good to go.

Process has a few steps that leads me to sit and run each one. Still looking into automating some of the steps yet, but the process even with multiple steps goes rather quick.

Thank you Very much for your posts and help!

Re: Stream capture from NBCOlympics

Sorry, I hadn't read the thread carefully enough to realize that you had already tried dealing with the fragments and were trying to avoid it.

I wasn't able to find the m3u/m3u8 style manifest for the most recently referenced video.

I'm confused why the f4m manifest results in only "keyframes" when used as a manifest url with AdobeHDS.php. I traced/watched tubedigger deal with that same f4m manifest (and only that manifest) and it is able to extract full urls for all of the audio and video fragments. I'm going to try to edit a local copy of the f4m manifest to see if it will work with the script directly.

Re: Stream capture from NBCOlympics

Quote:

Originally Posted by ehurdler (Post 64912)

chap, would you be able to explain how you got the syntax for this?...

When I sent a PM to chap to thank him for the ffmpeg parsing syntax, I asked him how he came up with it to which he responded, "interception traffic of mobile app(NBCOlympics)".

When ffmpeg doesn't work the way I expected, I kinda liked DownloadThemAll because of the GUI and being able to easily see the progress of each fragment being downloaded. However, since I was having problems joining them in their entirety due to AdobeHDS.php aborting on certain fragments, I'm now wondering if it is due to dTa downloading some fragments incompletely or incorrectly. I suppose I could try again and again to get all the fragments in their proper and complete form, but it's a real pain to know which fragments those are without time-consuming trial and error.

Wget - wow - I used to use that probably 20 years ago in order to save entire websites for offline viewing, but it never occurred to me to try to use it for grabbing these video fragments. Is there somewhere you can point me to that explains how to do this without downloading the entire internet?

Re: Stream capture from NBCOlympics

I have looked into the issue and pushed an update to fix it. here is the manifest content.

Code:

http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m

Code:

<?xml version="1.0" encoding="utf-8"?>

<manifest xmlns="http://ns.adobe.com/f4m/1.0" version="3.0">

        <streamType>recorded</streamType>

        <duration>163.067</duration>

        <bootstrapInfo profile="named" id="bootstrap_0">AAAAi2Fic3QAAAAAAAAAowAAAAPoAAAAAAACfPsAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAFIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAH0AAAAFIAAAAAAAJ40AAABCsAAABTAAAAAAACfPsAAAAAAA==</bootstrapInfo>

        <bootstrapInfo profile="named" id="bootstrap_1">AAABO2Fic3QAAAAAAAAAowAAAAPoAAAAAAACfOsAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAFIBAAAA9mFmcnQAAAAAAAAD6AAAAAAOAAAAAQAAAAAAAAAAAAAIWAAAAAIAAAAAAAAIWAAAB80AAAAQAAAAAAAAdY0AAAf7AAAAEQAAAAAAAH2IAAAHzQAAAB8AAAAAAADqvQAAB/sAAAAgAAAAAAAA8rkAAAfNAAAALgAAAAAAAV/tAAAH+wAAAC8AAAAAAAFn6QAAB80AAAA9AAAAAAAB1R4AAAf7AAAAPgAAAAAAAd0ZAAAHzQAAAEwAAAAAAAJKTgAAB/sAAABNAAAAAAACUkkAAAfNAAAAUgAAAAAAAnlKAAADoQAAAFMAAAAAAAJ86wAAAAAA</bootstrapInfo>

        <media label="video_3450" bitrate="3450" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(3450000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_3450_keyframes" bitrate="3450" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(3450000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_2200" bitrate="2200" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(2200000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_2200_keyframes" bitrate="2200" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_1400" bitrate="1400" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(1400000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_1400_keyframes" bitrate="1400" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_900" bitrate="900" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(900000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_900_keyframes" bitrate="900" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(900000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_600" bitrate="600" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(600000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_600_keyframes" bitrate="600" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(600000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_400" bitrate="400" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(400000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_400_keyframes" bitrate="400" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(400000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="video_200" bitrate="200" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(200000)/Fragments(video,format=f4m-f4f)"/>

        <media label="video_200_keyframes" bitrate="200" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(200000)/Keyframes(video,format=f4m-f4f)"/>

        <media label="audio_1_48" bitrate="48" type="audio" bootstrapInfoId="bootstrap_1" url="QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)" lang="en" alternate="true"/>

</manifest>

it contains two entries for each bitrate one for actual video stream and one for keyframes only. fragment url for "keyframe only fragments" was overwriting the actual video entry (due to same bitrate) resulting in downloading of keyframes only stream. also audio is being served as different stream instead of being muxed into video stream. when you run the updated script with debug switch it will show the following content.

Code:

php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -debug

Code:

Processing manifest info....

Manifest Entries:



 Bitrate URL

 3451    QualityLevels(3450000)/Keyframes(video,format=f4m-f4f)

 3450    QualityLevels(3450000)/Fragments(video,format=f4m-f4f)

 2201    QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)

 2200    QualityLevels(2200000)/Fragments(video,format=f4m-f4f)

 1401    QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)

 1400    QualityLevels(1400000)/Fragments(video,format=f4m-f4f)

 901     QualityLevels(900000)/Keyframes(video,format=f4m-f4f)

 900     QualityLevels(900000)/Fragments(video,format=f4m-f4f)

 601     QualityLevels(600000)/Keyframes(video,format=f4m-f4f)

 600     QualityLevels(600000)/Fragments(video,format=f4m-f4f)

 401     QualityLevels(400000)/Keyframes(video,format=f4m-f4f)

 400     QualityLevels(400000)/Fragments(video,format=f4m-f4f)

 201     QualityLevels(200000)/Keyframes(video,format=f4m-f4f)

 200     QualityLevels(200000)/Fragments(video,format=f4m-f4f)

 48      QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)



Quality Selection:

 Available: 3451 3450 2201 2200 1401 1400 901 900 601 600 401 400 201 200 48

 Selected : 3450

you can select your required stream by looking at url shown in debug info.

Video:

Code:

php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -quality 3450

Audio:

Code:

php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -quality 48

Remux:

Code:

ffmpeg -i Video.flv -i Audio.flv -c copy Final.mp4

PS: regarding the problem with converting some of the fragments downloded with DTA it's actually DTA's fault. sometimes it saves the error message sent by server instead of actual fragment data. also you don't have to rename the downloaded fragments. read the wiki again to understand how to do it properly.

Re: Stream capture from NBCOlympics

WOW, KSV, this is great! Thank you! Makes the process of capturing the non-event videos so much easier!!

Re: Stream capture from NBCOlympics

KSV
as always:cool:
respect for your work