Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)
-   Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)
-   -  

Olympic official streaming videos, i'm going crazy, i can't download it! help

 (http://stream-recorder.com/forum/showthread.php?t=13436)

pankyk86 08-20-2012 04:24 AM

Olympic official streaming videos, i'm going crazy, i can't download it! help


 
I'm trying and trying to download the streaming replay of official olympic channel on youtube but nothing to do. i've tried with a lot of programs, i don't know what i have to do for download for example a video like this:
http://www.youtube.com/watch?v=78btYWiv_xM
i hope you can help me
ps: i'ma windows 7 user

pankyk86 08-21-2012 12:42 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
no solution? anyone?

nwtim 08-21-2012 09:09 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
What have you tried so far?

Have you tried using ffmpeg using the following command line?

ffmpeg.exe -i URL -c copy out.ts

Using Firefox's web console (ctrl-shift-k), it seems to pick up the URL of:

"http://www.youtube.com/api/manifest/hls_playlist/id/efc6ed6168afff13/itag/96/source/yt_live_broadcast/ratebypass/yes/cmbypass/yes/playlist_type/CLEAN/maudio/1/pmbypass/yes/upn/4seIveUx-l8/sver/3/ip/24.18.126.11/ipbits/8/expire/1345624564/sparams/ip,ipbits,expire,id,itag,source,ratebypass,cmbypas s,playlist_type,maudio,pmbypass/signature/49A1DD88EFE092D59C5ABF52B187D4CEA6512E8B.428EB8078 7A258204F31E050620CCC3206B03A78/key/ck1/file/index.m3u8"

-----------------
I'm trying to get a bunch of video from the NBCOlympics site of raw streams not posted on their YouTube channel, and have had very little success trying to get the proper URLs.

So, I've made most of my recordings from this content (preferring 1080p or 720p) using Screen Recorder Suite. It works very well and easily, but the result is a lossy video.

You can actually use Video Downloadhelper to save some of the .mp2t files as they click by (a new file appears about every 3 or 4 seconds) and they'll play just fine in VLC (I rename them to .flv because they're actually flash video segments). The video quality is the best I've ever seen in my life, but the challenge is that a mere 10 minutes of video is saved in 130 files! and I have no idea how difficult it would be to merge all those files without issues.

I downloaded Video Streaming Recorder (another apowersoft product) and ran it on a trial basis. It grabs every single file that streams by without breaking a sweat and names them in sequential order. I "grabbed" a 4 hour long video of Rhythmic Gymnastics while I slept last night and woke up to over 3000 files. Video Streaming Recorder has a really nice set of conversion tools and it can even merge the files, but the trial mode limitation seems to disable any ability to properly join these files.

I've tried several pay-for tools, and virtually none of them even show these mp2t/flv files, but Video Streaming Recorder does.

If you find anything else that works I'd love to know, because the lossless video is to die for.

chap 08-22-2012 12:06 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 52862)
If you find anything else that works I'd love to know, because the lossless video is to die for.
example a video

nwtim 08-22-2012 02:41 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by chap (Post 52882)
example a video
I'd love to. Being new here, I haven't yet seen how most people share their videos. Got any tips on that? Meanwhile, PM me and I'll email you a short 2 or 3 second clip.

nwtim 08-22-2012 10:28 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 52862)
What have you tried so far?

Have you tried using ffmpeg using the following command line?

ffmpeg.exe -i URL -c copy out.ts

Using Firefox's web console (ctrl-shift-k), it seems to pick up the URL of:

"http://www.youtube.com/api/manifest/hls_playlist/id/efc6ed6168afff13/itag/96/source/yt_live_broadcast/ratebypass/yes/cmbypass/yes/playlist_type/CLEAN/maudio/1/pmbypass/yes/upn/4seIveUx-l8/sver/3/ip/24.18.126.11/ipbits/8/expire/1345624564/sparams/ip,ipbits,expire,id,itag,source,ratebypass,cmbypas s,playlist_type,maudio,pmbypass/signature/49A1DD88EFE092D59C5ABF52B187D4CEA6512E8B.428EB8078 7A258204F31E050620CCC3206B03A78/key/ck1/file/index.m3u8"
...
What I did was create a Downloads folder on a storage drive and added ffmpeg.exe to it and then created a getvideo.bat file with the above command line in it. The URL is something that will be user specific, so you need to find out what the long one is for the PC your working from. I used the web console in Firefox, but there are many tools that can do the job. Filter for the m3u8 filetype in the URL and paste it into the batch file with quotes around it. Once you've saved the batch file, open a command prompt window and manually navigate to the folder ffmpeg.exe and the batch file are in and type in the name of the batch file.

I just tried it with the video you mentioned, and it's downloading right now. It's not a fast download, but it's up to 137MB after just about 5 minutes. It's giving me a H264 AVC MPEG-4 file at 1920x1080 and 50 fps that looks amazing.

Good luck!

pankyk86 08-22-2012 05:43 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 52965)
What I did was create a Downloads folder on a storage drive and added ffmpeg.exe to it and then created a getvideo.bat file with the above command line in it. The URL is something that will be user specific, so you need to find out what the long one is for the PC your working from. I used the web console in Firefox, but there are many tools that can do the job. Filter for the m3u8 filetype in the URL and paste it into the batch file with quotes around it. Once you've saved the batch file, open a command prompt window and manually navigate to the folder ffmpeg.exe and the batch file are in and type in the name of the batch file.

I just tried it with the video you mentioned, and it's downloading right now. It's not a fast download, but it's up to 137MB after just about 5 minutes. It's giving me a H264 AVC MPEG-4 file at 1920x1080 and 50 fps that looks amazing.

Good luck!
can you explain step by step (for dummies :D ) , what is ffmpeg? how to download? i'm sorry but i'm not a pc genius

nwtim 08-22-2012 06:03 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53086)
can you explain step by step (for dummies :D ) , what is ffmpeg? how to download? i'm sorry but i'm not a pc genius
You can learn more about ffmpeg at http://ffmpeg.org/index.html. I downloaded mine (Windows) from http://ffmpeg.zeranoe.com/builds/. ffmpeg.exe and ffplay.exe can be placed in a folder of your choice and run from there with a command line - either from a command prompt in a window or from a batch file (which is how I use it).

Does that help?

Evic1981 08-22-2012 09:17 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
try to look at the instructions.

pankyk86 08-23-2012 11:22 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53093)
You can learn more about ffmpeg at http://ffmpeg.org/index.html. I downloaded mine (Windows) from http://ffmpeg.zeranoe.com/builds/.
i have to download just ffmep source code or all on this page?

pankyk86 08-23-2012 11:39 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
i downloaded builds "static" but this program is like arabic for me, i don't understand how to play, i click on ffmpeg icon but it appears just for 1 second a black window......

nwtim 08-23-2012 01:16 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53231)
i downloaded builds "static" but this program is like arabic for me, i don't understand how to play, i click on ffmpeg icon but it appears just for 1 second a black window......
ffmpeg.exe by itself will not do anything. That's why you're seeing a black window pop open and then close. It's an executable that needs arguments. You can see a listing of all the arguments it can deal with by opening a command window, navigate to a folder containing ffmpeg.exe, and type ffmpeg --help at the command prompt. It'll race by, so you could use the |more pipe (eg. ffmpeg --help|more) or just scroll back up to the command to read from the top.

I find the easiest way to deal with this kind of command line executable is to write a batch file, especially when dealing with lots of text like some of the URLs we have to deal with.

Do to this, I create a new text file with a line of text starting with the letters rem (remark) with a comment about the use of the file - usually just the subject of the video to be processed. On the next line I type in the ffmpeg command line that I wish to process. You don't have to, but I often put a last line in the file with the pause command so I can see any feedback in the command window before it closes. Then I save it as a batch file (eg. file.bat) and place it in the same folder as ffmpeg.exe. Make sure you have room for the new video file there, or else add a path to the output filename (spaces require that the output name is surrounded in quotes) where you know there's enough free space.

Code:
rem - 2012 Olympics Track & Field Men's 100 Meter Final
ffmpeg.exe -i "http://www.youtube.com/api/manifest/hls_playlist/id/.../key/ck1/file/index.m3u8" -c copy newvideo.ts
pause
Once you've got a batch file in the folder you've chose for ffmpeg and the video, you will not need to use the command prompt window. Just double-click on the batch file.

Does that make sense?

pankyk86 08-23-2012 03:01 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53237)
ffmpeg.exe by itself will not do anything. That's why you're seeing a black window pop open and then close. It's an executable that needs arguments. You can see a listing of all the arguments it can deal with by opening a command window, navigate to a folder containing ffmpeg.exe, and type ffmpeg --help at the command prompt. It'll race by, so you could use the |more pipe (eg. ffmpeg --help|more) or just scroll back up to the command to read from the top.

I find the easiest way to deal with this kind of command line executable is to write a batch file, especially when dealing with lots of text like some of the URLs we have to deal with.

Do to this, I create a new text file with a line of text starting with the letters rem (remark) with a comment about the use of the file - usually just the subject of the video to be processed. On the next line I type in the ffmpeg command line that I wish to process. You don't have to, but I often put a last line in the file with the pause command so I can see any feedback in the command window before it closes. Then I save it as a batch file (eg. file.bat) and place it in the same folder as ffmpeg.exe. Make sure you have room for the new video file there, or else add a path to the output filename (spaces require that the output name is surrounded in quotes) where you know there's enough free space.

Code:
rem - 2012 Olympics Track & Field Men's 100 Meter Final
ffmpeg.exe -i "http://www.youtube.com/api/manifest/hls_playlist/id/.../key/ck1/file/index.m3u8" -c copy newvideo.ts
pause
Once you've got a batch file in the folder you've chose for ffmpeg and the video, you will not need to use the command prompt window. Just double-click on the batch file.

Does that make sense?
thanks, it remains difficult to understand for me because i don't have a "great" experience in this :/ but for example in this video http://www.youtube.com/watch?v=78btYWiv_xM where is the ..api/manifest....? how can i take this link? i'm sorry, i'm a disaster!! maybe the best solution is someone who download video for me and then upload on a file sharging site :D

nwtim 08-23-2012 05:10 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53241)
thanks, it remains difficult to understand for me because i don't have a "great" experience in this :/ but for example in this video http://www.youtube.com/watch?v=78btYWiv_xM where is the ..api/manifest....? how can i take this link? i'm sorry, i'm a disaster!! maybe the best solution is someone who download video for me and then upload on a file sharging site :D
The URL for such programs often have an expiration date and they can be machine specific. I've found that to be the case with all the NBCOlympics videos that are served in 5-second chunks.

You won't be able to download/transcode (that's pretty much what ffmpeg is doing) the video without the proper URL, but there are numerous tools to do this. It's not completely trivial, as sometimes you have refresh the page and/or restart the video in order for the URL to be detected, but for these Olympics videos you're looking for the URL that ends in .m3u8. There are many ways to grab that URL. I happen to use the web console in Firefox (ctrl-shift-k) and enter m3u8 in the filter box in the upper right corner. The URL generally pops up right at the beginning of the video and that's all you need.

Don't give up - you're about to get this and then you'll be helping others.

pankyk86 08-23-2012 05:33 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53244)
The URL for such programs often have an expiration date and they can be machine specific. I've found that to be the case with all the NBCOlympics videos that are served in 5-second chunks.

You won't be able to download/transcode (that's pretty much what ffmpeg is doing) the video without the proper URL, but there are numerous tools to do this. It's not completely trivial, as sometimes you have refresh the page and/or restart the video in order for the URL to be detected, but for these Olympics videos you're looking for the URL that ends in .m3u8. There are many ways to grab that URL. I happen to use the web console in Firefox (ctrl-shift-k) and enter m3u8 in the filter box in the upper right corner. The URL generally pops up right at the beginning of the video and that's all you need.

Don't give up - you're about to get this and then you'll be helping others.
thanks for all your help and patience, i'm looking for the m3u8

pankyk86 08-23-2012 05:37 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
now i have this 3 :
http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 141ms]

[01:34:54.805] GET http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 150ms]

--
[01:35:10.479] GET http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 136ms]

what is the most important and what is the next step now?

quackeroats 08-24-2012 07:24 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Code:
ffmpeg -i http://www.youtube.com/api/manife
st/hls_variant/upn/BAK-PHP-zoU/sparams/id%2Cip%2Cipbits%2Cmaudio%2Cplaylist_type
%2Cpmbypass%2Cexpire/playlist_type/CLEAN/maudio/1/ip/209.197.20.248/ipbits/8/sig
nature/525A248DB3E26B0667C8A52013007E37840DC7FB.D52F09F1B969DED333C040DCE9E3E8E0
A1227A46/sver/3/source/yt_live_broadcast/expire/1345881156/key/yt1/pmbypass/yes/
id/016fd0b7d484de1f/file/index.m3u8 -c copy out.ts
ffmpeg version N-35709-g7d531e8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec 12 2011 13:50:00 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil    51. 32. 0 / 51. 32. 0
  libavcodec  53. 43. 0 / 53. 43. 0
  libavformat  53. 24. 0 / 53. 24. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter  2. 53. 0 /  2. 53. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[http @ 01CAF100] Key 'pb' not found.
[http @ 01CAF100] Error parsing options string ,ipbits,expire,id,itag,source,rat
ebypass,cmbypass,playlist_type,maudio,pmbypass/signature/6BA0E95DF9DA4DB1652A499
C146790C8CDE1082F.3843FAE1174BDB5492E92A7834A9E7E0B6E3691F/key/ck1/file/index.m3
u8
http://www.youtube.com/api/manifest/...arams/id%2Cip%
2Cipbits%2Cmaudio%2Cplaylist_type%2Cpmbypass%2Cexpire/playlist_type/CLEAN/maudio
/1/ip/209.197.20.248/ipbits/8/signature/525A248DB3E26B0667C8A52013007E37840DC7FB
.D52F09F1B969DED333C040DCE9E3E8E0A1227A46/sver/3/source/yt_live_broadcast/expire
/1345881156/key/yt1/pmbypass/yes/id/016fd0b7d484de1f/file/index.m3u8: Not enough
 space
anyone has this issue>?

nwtim 08-24-2012 10:46 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53247)
now i have this 3 :
http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 141ms]

[01:34:54.805] GET http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 150ms]

--
[01:35:10.479] GET http://www.youtube.com/api/manifest/...ile/index.m3u8 [HTTP/1.1 200 OK 136ms]

what is the most important and what is the next step now?
I usually take the last one, as it's the one that pops up the moment I click on the 1080p HD option of the quality gear in the video.

nwtim 08-24-2012 10:51 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by quackeroats (Post 53291)
Code:
ffmpeg -i http://www.youtube.com/api/manife
st/hls_variant/upn/BAK-PHP-zoU/sparams/id%2Cip%2Cipbits%2Cmaudio%2Cplaylist_type
%2Cpmbypass%2Cexpire/playlist_type/CLEAN/maudio/1/ip/209.197.20.248/ipbits/8/sig
nature/525A248DB3E26B0667C8A52013007E37840DC7FB.D52F09F1B969DED333C040DCE9E3E8E0
A1227A46/sver/3/source/yt_live_broadcast/expire/1345881156/key/yt1/pmbypass/yes/
id/016fd0b7d484de1f/file/index.m3u8 -c copy out.ts
ffmpeg version N-35709-g7d531e8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec 12 2011 13:50:00 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil    51. 32. 0 / 51. 32. 0
  libavcodec  53. 43. 0 / 53. 43. 0
  libavformat  53. 24. 0 / 53. 24. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter  2. 53. 0 /  2. 53. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[http @ 01CAF100] Key 'pb' not found.
[http @ 01CAF100] Error parsing options string ,ipbits,expire,id,itag,source,rat
ebypass,cmbypass,playlist_type,maudio,pmbypass/signature/6BA0E95DF9DA4DB1652A499
C146790C8CDE1082F.3843FAE1174BDB5492E92A7834A9E7E0B6E3691F/key/ck1/file/index.m3
u8
http://www.youtube.com/api/manifest/...arams/id%2Cip%
2Cipbits%2Cmaudio%2Cplaylist_type%2Cpmbypass%2Cexpire/playlist_type/CLEAN/maudio
/1/ip/209.197.20.248/ipbits/8/signature/525A248DB3E26B0667C8A52013007E37840DC7FB
.D52F09F1B969DED333C040DCE9E3E8E0A1227A46/sver/3/source/yt_live_broadcast/expire
/1345881156/key/yt1/pmbypass/yes/id/016fd0b7d484de1f/file/index.m3u8: Not enough
 space
anyone has this issue>?
I did when I ran out of free space on the drive I was writing to.

pankyk86 08-25-2012 09:25 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53296)
I usually take the last one, as it's the one that pops up the moment I click on the 1080p HD option of the quality gear in the video.
ok the last one, and now? what i have to do with this link? next step?

nwtim 08-25-2012 05:13 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53308)
ok the last one, and now? what i have to do with this link? next step?
Place ffmpeg in a folder you would like to download to and create a batch file there that looks like this:

Code:
rem Summary of the Games - London 2012 Olympic Games
ffmpeg.exe -i "http://www.youtube.com/api/manifest/hls_playlist/id/.../key/ck1/file/index.m3u8" -c copy "Summary of the Games - London 2012 Olympic Games.ts"
pause
after the -i is where you put the URL you've gotten inside quotes. Just make sure to remove any stuff before the http and after the m3u8.

Have fun.

pankyk86 08-25-2012 05:42 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53315)
Place ffmpeg in a folder you would like to download to and create a batch file there that looks like this:

Code:
rem Summary of the Games - London 2012 Olympic Games
ffmpeg.exe -i "http://www.youtube.com/api/manifest/hls_playlist/id/.../key/ck1/file/index.m3u8" -c copy "Summary of the Games - London 2012 Olympic Games.ts"
pause
after the -i is where you put the URL you've gotten inside quotes. Just make sure to remove any stuff before the http and after the m3u8.

Have fun.
thanks nwtim for all your help, you were great! i think i'm downloading now....the result is a .ts file
just another question, the last!! :) every video here has 3 different audio channels? how can i do to download the natural audio one? i find the way to download only the natural audio, but not the video with natural audio

nwtim 08-26-2012 01:38 AM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by pankyk86 (Post 53316)
thanks nwtim for all your help, you were great! i think i'm downloading now....the result is a .ts file
just another question, the last!! :) every video here has 3 different audio channels? how can i do to download the natural audio one? i find the way to download only the natural audio, but not the video with natural audio
I'm afraid you're ahead of me on the audio, as I'm only getting a single audio track - MPEG AAC stereo in English.

quackeroats 08-26-2012 10:37 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53297)
I did when I ran out of free space on the drive I was writing to.
ahhh... thanks i thought 30 Gb is enough cheers

quackeroats 08-27-2012 08:01 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by nwtim (Post 53297)
I did when I ran out of free space on the drive I was writing to.

ok it is not working as i hope it to be,
i tried to -ss 00:00:00 -t 00:10:00
and i still get the no space error, maybe space is not the issue?

nwtim 08-31-2012 10:43 PM

Re: Olympic official streaming videos, i'm going crazy, i can't download it! help


 
Quote:
Originally Posted by quackeroats (Post 53395)
ok it is not working as i hope it to be,
i tried to -ss 00:00:00 -t 00:10:00
and i still get the no space error, maybe space is not the issue?
Try it without the -ss and -t switches first. I've had inconsistent results with them and have been getting more consistent results when specifying the parameters in seconds rather than the xx:xx:xx format.


All times are GMT -6. The time now is 06:49 AM.