PDA

View Full Version : need help downloading from zipstreams.net


jimdays
11-12-2017, 06:00 PM
This website:
http://zipstreams.net/
has a huge number of videos. For example;
http://zipstreams.net/play.html?league=NHL&date=2017-11-11&id=54431503&pk=2017020249
There are always new videos being posted, and the older ones are not valid.
I want to download with only youtube-dl or ffmpeg on the command line. I already can download using a certain pay software that I have bought, so now I am only interested in downloading via youtube-dl or ffmpeg.
I can play the video in a browser network analyzer and get a master m3u8, but fails to download. If I use youtube-dl -F, I get a list of available resolutions, but trying any resolution entry will not download.

JonDough
11-12-2017, 06:30 PM
For example;
http://zipstreams.net/play.html?league=NHL&date=2017-11-11&id=54431503&pk=2017020249I already can download using a certain pay software that I have bought,

Like you, I am a big hockey fan. I would like to download these too. Please tell me the pay software to use. I tried these pay softwares - getflv, idm and tubedigger - but none worked. Thanks.

jimdays
11-13-2017, 06:01 AM
Like you, I am a big hockey fan.
Sorry, I could care less about hockey. I just like to download videos;it's a hobby. After I download them, I check to see if they play and then delete them. I paid for Tubedigger some time ago, and upon request, he will usually send a special version that will download from a particular website. After asking, he sent me a private version of Tubedigger that will download from zipstreams.net. For some reason, he asked me not to share the version publicly. So you will have to contact him through his website and request the software. If you want one particular video from zipstreams, I can download the the 640x360 (other choices are available) and post the link.
I'm still want to know how to download from zipstreams.net by using youtube-dl or ffmpeg. It is possible, I just need somebody to show me how to do it.

troller12
11-13-2017, 09:01 AM
Hi guys,

so you can download the video using ffmpeg tool but before you need to handle some steps manually I do explain now.

FFmpeg / streamlink are not working because they get bad request answer trying to access the keyfile links.Problem in this case is that the host to the keyfile is wrong which you can read in the complete-trimmed.m3u8 playlist of any your desired resolution & BANDWIDTH etc.If you follow manually the traffic in webconsole then you can see that the keyfile was loaded also from zipstreams.net host but if you download the playlist or you try to play it with ffmpeg / streamlink then you get the infos that the keyfile cant loaded from a other host mf.svc.nhl.com.So if you now just change the host name to zipstreams.net (only host name so the link is same) then you can access the keyfile links without any extra paramters.

Solution: Choose any stream quality you want from master playlist.I did choose this for my test...
1800K/1800_complete-trimmed.m3u8
....now download this entire playlist and load it into editor like Notepad ++.First change all keyfile hosts to original host of the site...
mf.svc.nhl.com
to
zipstreams.net
Use search & repleace function.As next you need to add the complete link to all short ts links at the beginning.
18/01/30.ts
to
http://hlslive-akc.med2.med.nhl.com/hdnts=exp=****/NHL_GAME_VIDEO_EDMNYR_M2_HOME_20171111_15103248963 68/1800K/18/01/31.ts
Do with all.Also you can use search & repleace function of Notepad++ like this...
18/
to
whole link + 18/
Do this 3 times for 18/ & 19/ & 20/ then you got all changed very quick.Now save it as new playlist file and copy this into your ffmpeg folder + CMD.exe and then start the cmd.exe from there and execute this command to start playing the stream in VLC.
ffmpeg.exe -protocol_whitelist "file,http,https,tcp,tls,crypto" -i "Hockey2.m3u8" -c:v copy -c:a copy -f mpegts - | C:\Programme\VideoLAN\VLC\vlc.exe -
In my case I named my edited playlist Hockey2.m3u8 so just use your name you gave it and also your VLC playerpath or output to download the video.If you did all correctly then you get this to see with ffmpeg...
https://img1.picload.org/image/droriwll/hockey..png
All working so far.In this case you also dont need to download the keyfile so you can access the keyfile links directly without any extra paramters.Just follow my steps and then you can download the video etc.

greetz

JonDough
11-13-2017, 12:43 PM
Thanks to troller and others that have responded by private message! Great help!

JonDough
11-13-2017, 12:50 PM
Sorry, I could care less about hockey

How much less could you care (https://www.merriam-webster.com/video/could-care-less-or-couldnt-care-less)? If you're saying that you like baseball, basketball and football better, please give hockey a try. We need more fans. You have the game downloaded already you could watch it and see how exciting the game is. It is not like watching golf that puts some people to sleep. Nothing against golf if that is your thing I play a couple of rounds myself every summer.

jimdays
11-13-2017, 09:07 PM
troller12,
OK, I got about 15% of the video downloaded and then failed. The reason is in Notepad++, you can't just replace 18/
to whole link + 18/
That will work for about the first 15% of the text file, until you get to
18/18/02.ts
19/18/00.ts
20/18/04.ts
and then Notepad++ will want to replace the second 18, which is incorrect. Likewise, the same problem will occur when you try to replace 19 and 20. How can you use Notepad++ to just replace the parts that need to be replaced? Please show details with screenshots. Also make sure your method will download the complete video without failing at some point (due to faulty editing of text file) before the entire video is downloaded.
One other thing: I don't want vlc to come up when I am downloading. I used this more simple command and it worked fine for downloading (no vlc in command line):
ffmpeg -i "Hockey2.m3u8" -c copy -bsf:a aac_adtstoasc output.mp4

JonDough
11-14-2017, 01:01 AM
There is a great program going by the name livestreamer that will play directly to vlc or download instead! It has a plugin named hlsvariant that works great with the hockey videos.

https://picload.org/thumbnail/drodidwr/hock_2.jpg (https://img3.picload.org/image/drodidwr/hock_2.jpg)

If you must be using ffmpeg the editing process outlined by troller works too. Edited file 1200.zip (http://www1.zippyshare.com/v/whzwzexp/file.html). I don't know how long the stuff gotten from zipstreams won't expire. I think maybe about 3 hours.

lesor
11-14-2017, 05:27 PM
That will work for about the first 15% of the text file, until you get to
18/18/02.ts
19/18/00.ts
20/18/04.ts
and then Notepad++ will want to replace the second 18, which is incorrect. Likewise, the same problem will occur when you try to replace 19 and 20. How can you use Notepad++ to just replace the parts that need to be replaced?
Try search & repleace function of Notepad++ like this...

18/18/ to whole link + 1*8/1*8/
18/19/ > ... 1*8/1*9/
18/20/ > ... 1*8/2*0/
...

19/18/ > ... 1*9/1*8/
...

20/20/ > ... 2*0/2*0/

18/ to whole link + 18/
19/ to whole link + 19/
20/ to whole link + 20/

1* to 1
2* to 2

https://s33.postimg.org/5qnt2tgkb/image.png (https://postimg.org/image/5qnt2tgkb/)

JonDough
11-14-2017, 09:15 PM
Try search & repleace function of Notepad++ like this...

18/18/ to whole link + 1*8/1*8/
18/19/ > ... 1*8/1*9/
18/20/ > ... 1*8/2*0/
...

19/18/ > ... 1*9/1*8/
...

20/20/ > ... 2*0/2*0/

18/ to whole link + 18/
19/ to whole link + 19/
20/ to whole link + 20/

1* to 1
2* to 2

https://s33.postimg.org/5qnt2tgkb/image.png (https://postimg.org/image/5qnt2tgkb/)

The easiest way is to search

005,1

and replace with

005,http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1510646404~acl=/*~id=nhlGatewayId:7294414~data=54431503~hmac=a03e5 5d122a615f6c5a0f0b6f1147ae7880d8339a2b8370edd4fff5 9abc06bd0/7c799250bbdc4ebe3982ea0506efa1c8/ls04/nhl/2017/11/11/NHL_GAME_VIDEO_EDMNYR_M2_HOME_20171111_15103248963 68/1200K/1


The "005,1" part has an embedded invisible line feed. This part should be copy/pasted from the source file into search and replace windows so that the line feed is not lost. Do not type in "005,1" into either the search or replace window.

Then repeat the above for these values

005,2


005,http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1510646404~acl=/*~id=nhlGatewayId:7294414~data=54431503~hmac=a03e5 5d122a615f6c5a0f0b6f1147ae7880d8339a2b8370edd4fff5 9abc06bd0/7c799250bbdc4ebe3982ea0506efa1c8/ls04/nhl/2017/11/11/NHL_GAME_VIDEO_EDMNYR_M2_HOME_20171111_15103248963 68/1200K/2


Two steps, done & done.

https://img3.picload.org/image/drooodrw/search_0.jpg

troller12
11-15-2017, 06:56 AM
Hi,

you can also use RegEx search mode in Notepad++
\w.+.ts
Let add the link + adding the matches...
http://....1800K/20/$&
For testing in realtime you can check any RegEx sites like this...
https://regexr.com/
https://regex101.com/
greetz

jimdays
11-18-2017, 04:08 PM
JonDough,
I tried your suggestion to use livestreamer. I am using livestreamer
(livestreamer-v1.12.2-win32-setup.exe) and I got a fresh m3u8 by playing the video in Chrome.
http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575003~hmac=2c808 b5b1469741463f3a262b9cb4cc9cc3d20699a88638d848fce6 8d97cf692/8924e48c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_20171118_15106796716 24/master_wired60.m3u8
I put that into livestreamer:
livestreamer "hlsvariant://http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575003~hmac=2c808 b5b1469741463f3a262b9cb4cc9cc3d20699a88638d848fce6 8d97cf692/8924e48c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_20171118_15106796716 24/master_wired60.m3u8" 360p

But I got download failure. Please look at my above code and following output and see if you can find what is wrong. Also, I don't want vlc to come up (as it did in your case). I just want to download the video. Please give me the code to put into livestreamer to do that.
Also to lessor, troller12, thank you for helping me edit the m3u8 file to download with ffmpeg. Your descriptions were too complicated for me to follow, but I was able to adapt your advice and come up with an easy solution for editing that I will describe in another post in this thread.
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\default>
C:\Documents and Settings\default>livestreamer "hlsvariant://http://hlslive-akc.
med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=545750
03~hmac=2c808b5b1469741463f3a262b9cb4cc9cc3d20699a 88638d848fce68d97cf692/8924e48
c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_2017
1118_1510679671624/master_wired60.m3u8" 360p
[cli][info] Found matching plugin stream for URL hlsvariant://http://hlslive-akc
.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575
003~hmac=2c808b5b1469741463f3a262b9cb4cc9cc3d20699 a88638d848fce68d97cf692/8924e4
8c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_201
71118_1510679671624/master_wired60.m3u8
[cli][info] Available streams: 216p (worst), 288p, 360p, 504p, 540p, 720p (best)

[cli][info] Opening stream: 360p (hls)
[stream.hls][error] Failed to create decryptor: Unable to open URL: https://mf.s
vc.nhl.com/ws/media/mf/v2.3/key/silk/mediaid/54575003/kid/70007937 (400 Client E
rror: Bad Request)
[cli][error] No data returned from stream

C:\Documents and Settings\default>

jimdays
11-18-2017, 05:39 PM
How to download zipstreams with ffmpeg. I tried this on several computers with different videos and it works.
Play zipstreams video with browser network analyser and copy m3u8 string to notepad (example pattern):
http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575003~hmac=2c808 b5b1469741463f3a262b9cb4cc9cc3d20699a88638d848fce6 8d97cf692/8924e48c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_20171118_15106796716 24/1800K/1800_complete-trimmed.m3u8

Use browser Chrome to download this m3u8. Should yield about 80kb text file called
1800_complete-trimmed.m3u8

Take above m3u8 string and cut off the end and add
\0
so you have:
http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575003~hmac=2c808 b5b1469741463f3a262b9cb4cc9cc3d20699a88638d848fce6 8d97cf692/8924e48c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_20171118_15106796716 24/1800K/\0
Save above string to notepad. I call this "http string".

Use notepad++ to replace all hostnames (mf.svc.nhl.com > zipstreams.net) in
1800_complete-trimmed.m3u8 (as shown in screenshot 1)
Save to notepad file. I called it 1800.txt

Go to:
https://regex101.com/
and drag 1800.txt file into
Test string box
Paste below into Regular Expression box
[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9]\.ts
Paste above "http string" into Substitution box
See screenshot 2, five red checkmarks indicate the important points.
Put cursor in bottom window, right click and select all so the the box turn blue, copy and
paste into new notepad file. You can rename to hockey.m3u8, put this file into folder that contains ffmpeg.
(open hockey.m3u8 in notepad++ and verify that all .ts strings have the http string (without \0) joined together.) example:
http://hlslive-akc.med2.med.nhl.com/hdnts=exp=1511085004~acl=/*~id=nhlGatewayId:6743919~data=54575003~hmac=2c808 b5b1469741463f3a262b9cb4cc9cc3d20699a88638d848fce6 8d97cf692/8924e48c510c791cdc783d4ad294ec06/ls04/nhl/2017/11/18/NHL_GAME_VIDEO_NYRCBJ_M2_HOME_20171118_15106796716 24/1800K/00/00/12.ts
and use this command to download on xp
ffmpeg -i "Hockey.m3u8" -c copy -bsf:a aac_adtstoasc output.mp4
this command on newer machines with newer ffmpeg;
ffmpeg -protocol_whitelist "file,http,https,tcp,tls,crypto" -i "Hockey.m3u8" -c copy -c:a copy output.mp4

https://files.videohelp.com/u/61125/zip1.jpg
https://files.videohelp.com/u/61125/zip2.jpg