How to dump youtube live TV stream to file using commandline?

How to dump youtube live TV stream to file using commandline?

 

I only know of only two ways to download a live youtube video:

The first way is using the full youtube URL using packet sniffing. This provides me a ridiculously long URL; which only works one or two times before I get a ERROR 403: Forbidden response.

For example:

The next way is using youtube-dl (python script). This way is very reliable and easy:

For example:
This allows me to use the share URL youtube provides. Unfortunately, this method starts to download the video starting from when the live TV event first began, instead of the current time of the live event. I dont know how to specify to download starting from current live broadcast time.

I was hoping someone could help me figure out the a consistent way to download this live TV stream starting from current time.

Re: How to dump youtube live TV stream to file using commandline?

 

try use

Re: How to dump youtube live TV stream to file using commandline?

 

Hi Chap, thank you so much for your help. I am able to use that URL. I have a question:

1. Is there a tutorial (or commandline app) which explains how to convert URLs like my original URL to a dependable URL? I prefer to learn to do this myself so I dont bother you for each URL :)

Sincerely,
MKANET

Re: How to dump youtube live TV stream to file using commandline?

 

open(user agent=ipad)
try use alternative

Re: How to dump youtube live TV stream to file using commandline?

 

I think maybe I didn't explain correctly originally; or, there is a miscommunication. I have always been able to get a full URL to download a youtube video using packet sniffer apps. The problem is it doesnt work for more than a few times before I get a HTTP error 403 Forbidden error message. It seems like the URL you provided (below) has already stopped working.

I am actually looking for a reliable/consistent way to download the stream (starting at the current live TV position).

The closest I can get is when using the youtube-dl python script that I've already been using; except it seems like its meant for non-live TV videos (starts download from when the live TV stream first started to broadcast instead of "current" broadcast live TV position in the web browser.

Re: How to dump youtube live TV stream to file using commandline?

 

simple use

Re: How to dump youtube live TV stream to file using commandline?

 

Yes, I tried both suggestions the first time you mentioned it. Unfortunately, it produces a URL that can only be used a few times. If I try to use it again, for example, the next day. The stream is no longer accessible using the URL produced by the below bookmarklet ...getting a HTTP error 403 Forbidden error message. It doesnt seem to be any better than the URLS I get via a protocol analyzer. The only way I can consistently get the stream is via youtube-dl; its unfortunately not really meant for live TV streams. Maybe there's something I'm missing?

Re: How to dump youtube live TV stream to file using commandline?

 

I wish there was a way to explain what I'm try to do more clearly.

It looks like Youtube.com is smart enough to not allow a static URL to access a video stream file more than a couple of times. Otherwise, it would be too easy. Im sure the URL would change pretty quicky. So, it doesnt even seem feasible to expect some kind of magic URL pointing to the every time. If there is, I haven't found a way to do that yet.

I am hoping someone was able to make a Windows command-line tool which can convert friendly short youtube URLs to a "direct" long URL to the file. Basically what the bookmarket above does, except in a self contained command-line app.

If there as a command-line converter that converted URLs like this:
http://www.youtube.com/watch?v=6QRKvM27Gxc

to

http://r16---sn-p5q7ynee.c.youtube.c...4DE5B4DF40314D

That would be great! Hopefully, Im not the only person who wants to do something like this.

Re: How to dump youtube live TV stream to file using commandline?

 

Not a answer really but you could look into seeing if there is a Windows port of clive or cclive ?
clive
cclive
That's if you are using Windows ?

Re: How to dump youtube live TV stream to file using commandline?

 

@evol

thanks for cclive
i downloaded cclive for windows but i could not download a youtube video?

that is the parameter


you can help





thx
hasomaso

Re: How to dump youtube live TV stream to file using commandline?

 

EDIT: There aren't any updated cclive builds for windows.
It seems youtube-dl is the only commandline downloader that has a Windows executable.

Re: How to dump youtube live TV stream to file using commandline?

 

Thanks, but I dont have cygwin; and, definitely not worth installing just for a shell script.

Re: How to dump youtube live TV stream to file using commandline?

 

I wasn't begging and certainly not from you.

Re: How to dump youtube live TV stream to file using commandline?

 

Suit yourself but it does work.

Re: How to dump youtube live TV stream to file using commandline?

 

No doubt it works for you, but if I was going to install an entire cygwin environment on my machine, I would have done it a long time ago; and certainly not for a single script. I do have cygwin1.dll on my machine for a few standalone exe's, but that's about it.

Re: How to dump youtube live TV stream to file using commandline?

 

I can write a commandline tool tomorrow that is similar to svnpenn's script, but it requires .NET Framework 2.0.

Re: How to dump youtube live TV stream to file using commandline?

 

That would be great. I already have .NET Framework 2.0. I appreciate it.

Re: How to dump youtube live TV stream to file using commandline?

 

@svnpenn5 I really do appreciate pointing me to babycygwin. I didnt even know there was such a thing.

One of the reasons why I was hoping for a windows native command-line tool was because it's much easier for me to integrate into my existing Windows batch files and apps; plus, the output would display natively in my Windows command prompt. I'm sure there are probably ways to further bridge the gap; but, it's hard to pass up a Windows native command-line tool if it's available.

BTW: I did try youtube.sh. I think Im doing something fundamentally wrong. But, it gave an error for me. It would be good to get this to work just in case the .NET solution doesnt come through.

Re: How to dump youtube live TV stream to file using commandline?

 

You need to get "Raw" version by clicking Raw link on GitHub.

Re: How to dump youtube live TV stream to file using commandline?

 

@svnpenn5 @Elgero, the whole purpose of needing this functionality in the first place is because I currently have Windows applications that depend on a external youtube plugin (looks for a user-defined Windows EXE,BAT, or CMD) to handle youtube streaming. Ive been using youtube-dl.exe for this, but unfortunately youtube-dl.exe doesnt handle live-tv correctly. So, I've been searching for a suitable Win32/Win64 youtube-dl.exe replacement.

With the solution that svnpenn5 so kindly created, I have no idea how to tell it what to do from a windows command prompt. Below, is the functionality I would need for this to work:

c:\>youtube youtube.com/watch?v=LHelEIJVxiE
46 1080p WebM vp8
37 1080p MP4 h.264 high
45 720p WebM vp8
102 720p WebM vp8 3D
22 720p MP4 h.264 high
84 720p MP4 h.264 3D
44 480p WebM vp8
35 480p FLV h.264 main
43 360p WebM vp8
100 360p WebM vp8 3D
34 360p FLV h.264 main
18 360p MP4 h.264 baseline
82 360p MP4 h.264 3D
5 240p FLV h.263
36 240p 3GP mpeg4 simple
17 144p 3GP mpeg4 simple
usage: /bin/youtube.sh [ITAG] [FILEPATH] URL

c:\>youtube 46 "E:\TV Recordings\video.mp4" youtube.com/watch?v=LHelEIJVxiE
--2013-01-26 11:52:27-- http://r3---sn-jvhj5nu-a5me.c.youtube.com/videoplayback?cp=U0hUTlJQUF9JTUNONF9KSlRJOlhPTUFVX 2ZKeTVu&cpn=HK55ba8vKaCJnpL2&expire=1359252843&fex p=916613%2C920704%2C912806%2C922403%2C922405%2C929 901%2C913605%2C925710%2C929104%2C920201%2C913302%2 C919009%2C911116%2C926403%2C910221%2C901451&id=2c7 7a5108255c621&ip=67.160.251.60&ipbits=8&itag=37&ke epalive=yes&key=yt1&ms=au&mt=1359229695&mv=m&newsh ard=yes&range=0-11886591&ratebypass=yes&signature=7F31C85ACEBA29AF 9EECFAC6232887A41BC66E34.0940C484FAA680D3086433326 0B3AA17629DB3AE&source=youtube&sparams=cp%2Cid%2Ci p%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cex pire&sver=3&upn=foxK64JaswQ
Resolving r3---sn-jvhj5nu-a5me.c.youtube.com (r3---sn-jvhj5nu-a5me.c.youtube.com)... 208.117.232.142
Connecting to r3---sn-jvhj5nu-a5me.c.youtube.com (r3---sn-jvhj5nu-a5me.c.youtube.com)|208.117.232.142|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 15745447 (15M) [video/webm] Saving to: `E:\TV Recordings\video.mp4'
91% [==================================> ] 14,364,940 1.30M/s eta 1s

@svnpenn5, I hope this isn't asking too much (at this point it is getting closer to begging :) ), but is there any way you could convert your batch script into a Windows executable (with builtin baby-cygwin) that works like the above example?

BTW, @Elgero, did you change your mind to make a .net based script that does this?

Thanks so much,
MKANET

Re: How to dump youtube live TV stream to file using commandline?

 

Can you give me an example of a livestream?

A live stream like this?

http://www.youtube.com/user/Aiekillu...E&feature=plcp

:edit:

Changed live stream. Previous one went offline.

Re: How to dump youtube live TV stream to file using commandline?

 

Like this one:
http://www.youtube.com/watch?v=L7ird1HeEjw

Edit: Just to be clear, youtube-dl CAN download live-tv streams like my example just fine, except they start downloading from the very beginning of the stream from several days ago (in case you didnt see this in my earlier posts in this thread). The temporary link produced be svnpenn (and my protocol analyzer) produce a url that downloads the livetv stream at the CURRENT live broadcat time; which is the right starting point.

Re: How to dump youtube live TV stream to file using commandline?

 

Try this. I've only tested it on a few streams, but seems to work ok.

<deleted. new url below>

Re: How to dump youtube live TV stream to file using commandline?

 

@Elgero, thank you so much for doing this!! I can confirm that it works for live TV likes such as the ones here: http://www.youtube.com/live/all. That's a big step closer to get it to work like svnpenn's script; which supports livetv and non-livetv full URLs.

Example usage:

c:\>
c:\>YouTubeDL.exe http://youtube.com/watch?v=LHelEIJVxiE

Available formats for stream:
46 1080p WebM vp8
37 1080p MP4 h.264 high
45 720p WebM vp8
102 720p WebM vp8 3D
22 720p MP4 h.264 high
84 720p MP4 h.264 3D
44 480p WebM vp8
35 480p FLV h.264 main
43 360p WebM vp8
100 360p WebM vp8 3D
34 360p FLV h.264 main
18 360p MP4 h.264 baseline
82 360p MP4 h.264 3D
5 240p FLV h.263
36 240p 3GP mpeg4 simple
17 144p 3GP mpeg4 simple

usage: YouTubeDL.exe [Format] [Filepath] [url]

c:\>
c:\>YouTubeDL.exe 46 "E:\TV Recordings\video.mp4" http://youtube.com/watch?v=LHelEIJVxiE

Saving as: E:\TV Recordings\video.mp4
Downloading Youtube Stream: 237 KB - 00:00:02

Thanks again for all your trouble. A tool like this can easily get very popular; since it would have more usefulness than even the popular opensource project for "youtube-dl"

Re: How to dump youtube live TV stream to file using commandline?

 

Try this one.

https://rapidshare.com/files/3998189519/YouTubeDL.zip

Re: How to dump youtube live TV stream to file using commandline?

 

hi,

youtubedl downloaded youtube videos ok
youtubedl also plays videos with VLC..?

I tested the same way

but does not work???

can you help me please




thx
hasomaso

Re: How to dump youtube live TV stream to file using commandline?

 

This is almost perfect! There is a couple of very minor items I noticed:

Sorry, in advance that Issue 1 is so long. Its very hard for me to describe such a seemingly simple symptom:

Item 1: For example, while a media file is still downloading (20 seconds after starting the download), if I try to play it back in my media player, the video will playback up until 20 seconds, then get stuck.

It seems like there is a value/attribute saved in the media file which stores the media file's estimated length/size of the file. A couple of my media players seems to look at that value; and, stops at the file-length it reads when it first started playing the media file back.

I am hoping you might be able to remove this attribute in the media file so the player wont be able to read the length/size of the file, so it can continue playback back a growing media file.

Youtube-dl.exe already knows how to handle this issue and displays a NO-ETA for live TV streams:
[download] 0.1% of 2.00G at 66.15k/s ETA --:--
So, none of my players have a problem playback back liveTV streams.


Item 2: This is a very very minor issue, but if possible, could you make it so if the format is not specified by the user, it will automatically select the highest quality stream? This is also what Youtube-dl.exe does.


Issue 3: This is also a very minor issue, but if possible, could you also allow the full URL to be specified by the user? So, "http://www.youtube.com/watch?v=v_UyVmITiYQ" could be used instead of only "v_UyVmITiYQ". My applications use the full URL as their URL variable. No worries if this is too hard to do.


Thanks for all your time.. If possible, I would like to offer a donation via paypal for your time.

Re: How to dump youtube live TV stream to file using commandline?

 

I would also like the supportet youtubedl vlc

Re: How to dump youtube live TV stream to file using commandline?

 

Item 1:

FLV videos can contain metadata at the beginning with information about the video file, like the duration, but this is not the case with live streams from YouTube. Your video player probably reads the timestamp of the last audio/video frame and gets the duration that way.

I'm not sure what the problem is, but I can change the file permissions in the code (it's set to write only now) and maybe that will solve the issue. VLC doesn't have any problems with playing the video while downloading it and keeps on playing, so you might want to try that.

The 2.00G that youtube-dl shows is simply the content-length header returned from the youtube server. When downloading a normal file it returns the size of the file in bytes. With a live stream it returns 2147483648 (2GB).

Item 2:

Yes.

Item 3:

Yes.

I'll do it tomorrow.

Re: How to dump youtube live TV stream to file using commandline?

 

I need to modify it to be able to write to stdout.

I'll do it tomorrow.

Re: How to dump youtube live TV stream to file using commandline?

 

Thanks for taking a look at this. Unfortunately, it's not possible for me to change my player. It's my media center software which runs my entire home's TV/Audio/Video in every room (SageTV). It has a plugin to allow me to do whatever I want using command-line; which I've used youtube-dl.exe for. Whatever the issue is, it doesnt look like its by accident that youtube-dl.exe is able able produce media file that's playable until the "real" end of the media file; which could go on indefinitely.

I have a feeling this might not be a file permissions issue. Whatever the issue is, it only affects liveTV videos produced with your executable, not video clips with a defined ending. if it was a permissions issue the same behavior would occur with video clips as well. For example, if I start download a 10 minute video; and, start playing it back in SageTV after 30 seconds, it will continue playing it back until the very end. This isn't true doing the same thing with liveTV clips. Do you think youtube-dl changes the livetv file attribute to have a fake, unobtainable very long video clip lenth such as a 10 hour video? that way the play will try to play it all the way until the end no matter what.

Anyway.. thanks a million for helping us!

Re: How to dump youtube live TV stream to file using commandline?

 

Here's a new version.

https://rapidshare.com/files/18912184/YouTubeDL.zip

- Highest quality stream will be downloaded when format argument is omitted.
- Full url must be specified.
- Stream can be redirected to stdout and played with VLC.

I haven't done anything about the live stream getting stuck (yet). I have no idea what youtube-dl does and I'm not going to dig through the source code of youtube-dl or experiment for hours trying to find that out. If somebody can tell me how youtube-dl does it, then I might be able to fix it.

Re: How to dump youtube live TV stream to file using commandline?

 

woww thanks

Re: How to dump youtube live TV stream to file using commandline?

 

Thank you!!!

Elgero, it works perfectly! After further testing, it look like the "stuck video" issue isn't limited to your utility. I was wrong, it happens to youtube-dl as well; including raw youtube download using wget.

The problem I have with my player looks like it presumes the media file doesnt grow; and, only reads the file length when it first starts playback. If the player looks at the file size; then I doubt there's anything that can be done.

Having said that, great job! Your utility is actually more functional than the original youtube-dl utility! You may see lots of people wanting to use this. :)

Re: How to dump youtube live TV stream to file using commandline?

 

Elgero, I hope you dont mind. I shared your tool with youtube-dl users who need better handling of live TV video downloads.

BTW: I'm not sure if this is fixable or not. But, on some live TV videos, there is no format detected. The same thing happens with youtube-dl.exe

Re: How to dump youtube live TV stream to file using commandline?

 

Yes, that's fine.

I don't think it's easy to fix. Not all live streams are from YouTube. Some live streams are from other websites, but embedded on the YouTube website. The white house streams are from live.whitehouse.gov (Akamai) and as far as I know they provide rtmp streams.

Re: How to dump youtube live TV stream to file using commandline?

 

hello Elgero,

thank you for the great youtube-dl tool, works perfect mate ;)

can you please expand the tool and a few streaming sites integrate?

the sites are free...


for example this sites

1. http://xhamster.com/

2. http://www.thenewporn.com

3. http://www.tnaflix.com/

4. http://www.youporn.com/





thx Bro.
hasomaso

Re: How to dump youtube live TV stream to file using commandline?

 

Sorry, that's not going to happen. I spent several hours making this tool and it has only been downloaded 11 times in almost 3 weeks, so there isn't much interest in this tool. If I add those websites, then I'm basically just doing it for you and I'm not going to do that.

There are already a lot of tools out there that can download from all these websites.

Re: How to dump youtube live TV stream to file using commandline?

 

hello Elgero,

i can understand you well, but you can force anyone
of course there are many download tools on the internet, but youtube-dl has certain something.

I would be happy, if you are integrating the pages in your great tool

my email you can find under my profile



thx
hasomaso

Re: How to dump youtube live TV stream to file using commandline?

 

Like I said, I won't do it. It's a 'YouTube' downloader that was designed for downloading from YouTube only. Adding other websites to the tool requires me to rewrite most of the tool and already nobody is interested. And if one of these websites changes something, then I am forced to update it. I'm really not interested in writing or updating a tool just for a handful of people, while I'm not using it myself.

Maybe you can contact the author of youtube-dl (another console youtube downloader), to add support for those websites. He already added xvideos.com, so he might add other websites as well.