Audio/video stream recording forums

Attention Visitor:
You may have to register or log in before you can post:
  • Click the register link to sign up.
  • Registered members please fill in the form below and click the "Log in" button.
To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Audio/video stream recording forums > Streaming media recording forum > Video stream recording
Register FAQ Members List Calendar Mark Forums Read

Reply Post New Thread
 
Thread Tools Display Modes
  #31  
Old 02-22-2014, 03:17 PM
ehurdler ehurdler is offline
Senior Member
 
Join Date: Feb 2014
Location: NY
Posts: 126
ehurdler is on a distinguished road
Default

Re: Stream capture from NBCOlympics


Quote:
Originally Posted by chap View Post
try simple
Code:
ffmpeg -i "http://olystreameast.nbcolympics.com/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/QualityLevels(56000)/Manifest(audio_en_0,format=m3u8-aapl,filtername=vodcut)" -i "http://olystreameast.nbcolympics.com/vod/8ea27d49-ddab-48cd-8d4e-89eecd346124/ladies--short-program0219054304.ism/QualityLevels(3450000)/Manifest(video,format=m3u8-aapl,filtername=vodcut)" -c copy t.ts
chap, would you be able to explain how you got the syntax for this? I am trying to capture some of the non-sports videos and the syntax you provided does not work for those. Example, this link: http://www.nbcolympics.com/video/exa...act-space-race

If I navigate to what comes up to be the manifest file, I can see all the options for video stream index, and audio stream index. I just can't get the proper code/syntax to reproduce anything that will stream download with ffmpeg. Quick copy/paste from manifest is:
Code:
<SmoothStreamingMedia MajorVersion="2" MinorVersion="0" Duration="11395717666">
  <StreamIndex Type="video" Chunks="570" QualityLevels="7" MaxWidth="1280" MaxHeight="720" DisplayWidth="1280" DisplayHeight="720" Url="QualityLevels({bitrate})/Fragments(video={start time})">
    <QualityLevel Index="0" Bitrate="3450000" FourCC="H264" MaxWidth="1280" MaxHeight="720" CodecPrivateData="0000000127640029AC723014016EC04400000FA40003A983888001A5240001A524EF7B80F844229C0000000128FFBCB0" NominalBitrate="4485000"/>
    <QualityLevel Index="1" Bitrate="2200000" FourCC="H264" MaxWidth="960" MaxHeight="540" CodecPrivateData="00000001274D401FB9181E022FDE022000007D20001D4C1C0C000432380008647DEF701F088453800000000128FFBC80" NominalBitrate="2860000"/>
    <QualityLevel Index="2" Bitrate="1400000" FourCC="H264" MaxWidth="704" MaxHeight="396" CodecPrivateData="00000001274D401FB91816067F7808800001F4800075307030002AB9800155CF7BDC07C22114E00000000128FFBC80" NominalBitrate="1820000"/>
    <QualityLevel Index="3" Bitrate="900000" FourCC="H264" MaxWidth="640" MaxHeight="360" CodecPrivateData="000000012742C01EB9101405FF2E022000007D20001D4C1C480036EE0006DDDDEF701F08846A0000000128DE3C80" NominalBitrate="1170000"/>
    <QualityLevel Index="4" Bitrate="600000" FourCC="H264" MaxWidth="512" MaxHeight="288" CodecPrivateData="000000016742C01EB6808025B01100000303E90000EA60E06000927C00124FE6A200F8B1750000000168CE3C80" NominalBitrate="780000"/>
    <QualityLevel Index="5" Bitrate="400000" FourCC="H264" MaxWidth="340" MaxHeight="192" CodecPrivateData="000000016742C01EB681619E7F011000003E90000EA60E280030D40061AE6A200F8B17500000000168CE3C80" NominalBitrate="520000"/>
    <QualityLevel Index="6" Bitrate="200000" FourCC="H264" MaxWidth="340" MaxHeight="192" CodecPrivateData="000000016742C01EB681619E7F011000003E90000EA60E060030D40061AE6A200F8B17500000000168CE3C80" NominalBitrate="260000"/>
    <c n="0" d="20020000"/>
...
...
...
    <c n="568" d="20020000"/>
    <c n="569" d="4337666"/>
</StreamIndex>
  <StreamIndex Type="audio" Name="audio_1" Chunks="570" QualityLevels="1" Language="eng" Url="QualityLevels({bitrate})/Fragments(audio_1={start time})">
    <QualityLevel Index="0" Bitrate="48000" FourCC="AACH" SamplingRate="22050" Channels="2" BitsPerSample="16" PacketSize="4" AudioTag="255" CodecPrivateData="139056E5A0"/>
    <c n="0" d="21362358"/>
...
...
...
    <c n="568" d="19969161"/>
    <c n="569" d="2786395"/>
  </StreamIndex>
</SmoothStreamingMedia>
Any help would be greatly appreciated. Thank you.
Reply With Quote
  #32  
Old 02-22-2014, 09:37 PM
blimey blimey is offline
Former Member
 
Join Date: Jul 2006
Posts: 286
blimey is on a distinguished road
Default

Re: Stream capture from NBCOlympics


Quote:
Originally Posted by ehurdler View Post
chap, would you be able to explain how you got the syntax for this? I am trying to capture some of the non-sports videos and the syntax you provided does not work for those. Example, this link:
Code:
h..p://www.nbcolympics.com/video/examining-impact-space-race
The fragment urls still work just fine. It is almost as easy to download all the fragments with a batch downloader and then use the script to join the fragments, and ffmpeg to mux. The sync came out fine here.

For example, for the the link you reference above;
Code:
video:

h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(3450000)/Fragments(video,format=f4m-f4f)Seg1-Frag1
.
.
.
h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(3450000)/Fragments(video,format=f4m-f4f)Seg1-Frag569

audio:

h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)Seg1-Frag1
.
.
.
h..p://olyvodwest.nbcolympics.com/vod/91e6dfa5-213a-4d90-9e82-5a6aa6fec769/216_XX_2461_SPACE_RACE.ism/QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)Seg1-Frag569
After running the script seperately on each set of fragments, mux the audio and video with ffmpeg:
Code:
ffmpeg -i Joinedvideo.flv -i Joinedaudio.flv -c copy video_audio.mp4

Using IDM batch download for the fragments seems faster than ffmpeg. Also note, for videos that don't require any provider authentication, like the one you reference here, TD will download and join the fragments for both the video and the audio (all quality levels offered), you just have to mux them.

Last edited by blimey : 02-22-2014 at 11:54 PM.
Reply With Quote
  #33  
Old 02-22-2014, 10:16 PM
klatwork klatwork is offline
Junior Member
 
Join Date: Nov 2011
Posts: 14
klatwork is on a distinguished road
Default

Re: Stream capture from NBCOlympics


one word: TUBEDIGGER
Reply With Quote
  #34  
Old 02-22-2014, 10:34 PM
blimey blimey is offline
Former Member
 
Join Date: Jul 2006
Posts: 286
blimey is on a distinguished road
Default

Re: Stream capture from NBCOlympics


TD is great, but it seems to only offer a full manifest (with all the quality selections) for public videos.

For videos requiring provider authentication, it would only download one video quality (not the highest), and didn't offer to download audio at all. If you had a different experience with non-public (authentication required) videos using TD, please share.
Reply With Quote
  #35  
Old 02-23-2014, 09:38 AM
ehurdler ehurdler is offline
Senior Member
 
Join Date: Feb 2014
Location: NY
Posts: 126
ehurdler is on a distinguished road
Default

Re: Stream capture from NBCOlympics


Thank you Blimey!

I had downloaded the individual fragments originally with DownThemAll for that video link I had posted, Space Race video.... Well that had worked, but seemed to be too long of a process. I had seen what chap wrote a few posts before, and that was working for capturing the event/sports. As I stated it did not work for capturing the features/news links off the site.

I had done some more searching after your most recent post. I installed wget, made a temp url.txt list. I keep quickly can now copy/paste the correct url in the list, add or remove lines to count for more or less fragment files. I can quickly run wget with the url list for audio and video fragments, very quick to download them into separate folders. I have to run a renaming app to rename to 1.f4f, 2.f4f, etc. etc. I am on a Macintosh so I use A Better File Rename 8. I am looking into how to use wget to download fragments and rename all in the same process.

Run
Code:
php AdobeHDS.php
for each type, audio/video. Gives me the Joined.flv for both, rename to JoinedAudio.flv, JoinedVideo.flv. Quickly run with ffmpeg as you stated
Code:
ffmpeg -i Joinedvideo.flv -i Joinedaudio.flv -c copy video_audio.mp4
I am good to go.

Process has a few steps that leads me to sit and run each one. Still looking into automating some of the steps yet, but the process even with multiple steps goes rather quick.

Thank you Very much for your posts and help!
Reply With Quote
  #36  
Old 02-23-2014, 11:56 AM
blimey blimey is offline
Former Member
 
Join Date: Jul 2006
Posts: 286
blimey is on a distinguished road
Default

Re: Stream capture from NBCOlympics


Sorry, I hadn't read the thread carefully enough to realize that you had already tried dealing with the fragments and were trying to avoid it.

I wasn't able to find the m3u/m3u8 style manifest for the most recently referenced video.

I'm confused why the f4m manifest results in only "keyframes" when used as a manifest url with AdobeHDS.php. I traced/watched tubedigger deal with that same f4m manifest (and only that manifest) and it is able to extract full urls for all of the audio and video fragments. I'm going to try to edit a local copy of the f4m manifest to see if it will work with the script directly.
Reply With Quote
  #37  
Old 02-23-2014, 04:36 PM
nwtim nwtim is offline
Member
 
Join Date: Aug 2012
Posts: 59
nwtim is on a distinguished road
Default

Re: Stream capture from NBCOlympics


Quote:
Originally Posted by ehurdler View Post
chap, would you be able to explain how you got the syntax for this?...
When I sent a PM to chap to thank him for the ffmpeg parsing syntax, I asked him how he came up with it to which he responded, "interception traffic of mobile app(NBCOlympics)".

When ffmpeg doesn't work the way I expected, I kinda liked DownloadThemAll because of the GUI and being able to easily see the progress of each fragment being downloaded. However, since I was having problems joining them in their entirety due to AdobeHDS.php aborting on certain fragments, I'm now wondering if it is due to dTa downloading some fragments incompletely or incorrectly. I suppose I could try again and again to get all the fragments in their proper and complete form, but it's a real pain to know which fragments those are without time-consuming trial and error.

Wget - wow - I used to use that probably 20 years ago in order to save entire websites for offline viewing, but it never occurred to me to try to use it for grabbing these video fragments. Is there somewhere you can point me to that explains how to do this without downloading the entire internet?
Reply With Quote
  #38  
Old 02-24-2014, 02:18 AM
KSV KSV is offline
Senior Member
 
Join Date: Apr 2011
Posts: 853
KSV is on a distinguished road
Cool

Re: Stream capture from NBCOlympics


I have looked into the issue and pushed an update to fix it. here is the manifest content.

Code:
http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m
Code:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0" version="3.0">
	<streamType>recorded</streamType>
	<duration>163.067</duration>
	<bootstrapInfo profile="named" id="bootstrap_0">AAAAi2Fic3QAAAAAAAAAowAAAAPoAAAAAAACfPsAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAFIBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAH0AAAAFIAAAAAAAJ40AAABCsAAABTAAAAAAACfPsAAAAAAA==</bootstrapInfo>
	<bootstrapInfo profile="named" id="bootstrap_1">AAABO2Fic3QAAAAAAAAAowAAAAPoAAAAAAACfOsAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAFIBAAAA9mFmcnQAAAAAAAAD6AAAAAAOAAAAAQAAAAAAAAAAAAAIWAAAAAIAAAAAAAAIWAAAB80AAAAQAAAAAAAAdY0AAAf7AAAAEQAAAAAAAH2IAAAHzQAAAB8AAAAAAADqvQAAB/sAAAAgAAAAAAAA8rkAAAfNAAAALgAAAAAAAV/tAAAH+wAAAC8AAAAAAAFn6QAAB80AAAA9AAAAAAAB1R4AAAf7AAAAPgAAAAAAAd0ZAAAHzQAAAEwAAAAAAAJKTgAAB/sAAABNAAAAAAACUkkAAAfNAAAAUgAAAAAAAnlKAAADoQAAAFMAAAAAAAJ86wAAAAAA</bootstrapInfo>
	<media label="video_3450" bitrate="3450" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(3450000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_3450_keyframes" bitrate="3450" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(3450000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_2200" bitrate="2200" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(2200000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_2200_keyframes" bitrate="2200" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_1400" bitrate="1400" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(1400000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_1400_keyframes" bitrate="1400" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_900" bitrate="900" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(900000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_900_keyframes" bitrate="900" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(900000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_600" bitrate="600" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(600000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_600_keyframes" bitrate="600" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(600000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_400" bitrate="400" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(400000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_400_keyframes" bitrate="400" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(400000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="video_200" bitrate="200" type="video" bootstrapInfoId="bootstrap_0" url="QualityLevels(200000)/Fragments(video,format=f4m-f4f)"/>
	<media label="video_200_keyframes" bitrate="200" type="video-keyframe-only" bootstrapInfoId="bootstrap_0" url="QualityLevels(200000)/Keyframes(video,format=f4m-f4f)"/>
	<media label="audio_1_48" bitrate="48" type="audio" bootstrapInfoId="bootstrap_1" url="QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)" lang="en" alternate="true"/>
</manifest>
it contains two entries for each bitrate one for actual video stream and one for keyframes only. fragment url for "keyframe only fragments" was overwriting the actual video entry (due to same bitrate) resulting in downloading of keyframes only stream. also audio is being served as different stream instead of being muxed into video stream. when you run the updated script with debug switch it will show the following content.
Code:
php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -debug
Code:
Processing manifest info....
Manifest Entries:

 Bitrate URL
 3451    QualityLevels(3450000)/Keyframes(video,format=f4m-f4f)
 3450    QualityLevels(3450000)/Fragments(video,format=f4m-f4f)
 2201    QualityLevels(2200000)/Keyframes(video,format=f4m-f4f)
 2200    QualityLevels(2200000)/Fragments(video,format=f4m-f4f)
 1401    QualityLevels(1400000)/Keyframes(video,format=f4m-f4f)
 1400    QualityLevels(1400000)/Fragments(video,format=f4m-f4f)
 901     QualityLevels(900000)/Keyframes(video,format=f4m-f4f)
 900     QualityLevels(900000)/Fragments(video,format=f4m-f4f)
 601     QualityLevels(600000)/Keyframes(video,format=f4m-f4f)
 600     QualityLevels(600000)/Fragments(video,format=f4m-f4f)
 401     QualityLevels(400000)/Keyframes(video,format=f4m-f4f)
 400     QualityLevels(400000)/Fragments(video,format=f4m-f4f)
 201     QualityLevels(200000)/Keyframes(video,format=f4m-f4f)
 200     QualityLevels(200000)/Fragments(video,format=f4m-f4f)
 48      QualityLevels(48000)/Fragments(audio_1,format=f4m-f4f)

Quality Selection:
 Available: 3451 3450 2201 2200 1401 1400 901 900 601 600 401 400 201 200 48
 Selected : 3450
you can select your required stream by looking at url shown in debug info.

Video:
Code:
php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -quality 3450
Audio:
Code:
php AdobeHDS.php -manifest "http://olyvodeast.nbcolympics.com/vod/c2940c3f-3bd7-4a0c-8101-fd847ee226ba/KRII_VOD_OZ0221_AllAround_Sochi_Today_Show.ism/manifest(format=f4m-f4f).f4m" -quality 48
Remux:
Code:
ffmpeg -i Video.flv -i Audio.flv -c copy Final.mp4
PS: regarding the problem with converting some of the fragments downloded with DTA it's actually DTA's fault. sometimes it saves the error message sent by server instead of actual fragment data. also you don't have to rename the downloaded fragments. read the wiki again to understand how to do it properly.

Last edited by KSV : 02-24-2014 at 03:02 AM.
Reply With Quote
  #39  
Old 02-24-2014, 07:23 AM
ehurdler ehurdler is offline
Senior Member
 
Join Date: Feb 2014
Location: NY
Posts: 126
ehurdler is on a distinguished road
Default

Re: Stream capture from NBCOlympics


WOW, KSV, this is great! Thank you! Makes the process of capturing the non-event videos so much easier!!
Reply With Quote
  #40  
Old 02-24-2014, 11:50 AM
chap chap is offline
Senior Member
 
Join Date: Feb 2011
Location: Ukraine
Posts: 1,165
chap is on a distinguished road
Default

Re: Stream capture from NBCOlympics


KSV
as always
respect for your work
Reply With Quote
Reply Post New Thread
Tags: , ,



Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -6. The time now is 11:41 PM.


Powered by All-streaming-media.com; 2006-2011
vB forum hacked with Zoints add-ons