Help capturing from Fox.com.

[ehurdler]

Let me say, I have found a way to capture videos from fox.com since they made changes to their website. The process is rather long, but again, it does work. I was wondering if someone, anyone may be able to help to make the process easier and or faster.

Let me state, youtube-dl does work....but it also grabs the commercials and adds to final download.

Ok so here is my process.

This is for Fox's new show The Orville - s01e02 - https://www.fox.com/watch/3cecd74bcf...84a67c2bb1648/

Load video in question.
Got to Tools->Web Developer->Network
Use All Tab
Filter URLs - Search for link.

You will find for example:

https://link.theplatform.com/s/fox-dcg/media/rTAuVMt2lj_p?format=redirect&formats=m3u&assetTypes=uplynk-clean:uplynk-ivod-west:uplynk-ivod-mountain:uplynk-ivod-east:uplynk-ivod&sitesection=app.dcg-foxnow/web/fox&adenv=fw_prod

Take the number string after /media/ in the url, in this case “rTAuVMt2lj_p” add between /media/ and ?

https://link.theplatform.com/s/fox-dcg/media/rTAuVMt2lj_p?mbr=true&manifest=m3u&formats=MPEG4,FLV,M3U&format=SMIL

Paste in browser. Hit enter. Will download a file “rTAuVMt2lj_p”.
Open this file in a text editor.
Now look for

<param name="testPlayerUrl" value="http://content.uplynk.com/player5/6BMB892QnCASKGD1s1jgmIsa.html"/>

Change User Agent to iPhone 3, also Disable Flash. Now Paste url into browser, start playing the video.

Once starts streaming you can either save video of choice with “Video DownloadHelper”
Or you can copy the string from the video you would download with Video Download Helper into ffmpeg
Examples below:

ffmpeg -i "https://content-auso3.uplynk.com/6c68ec3ffc3c4df7972591ba15629fcf/j.m3u8?cid=6c68ec3ffc3c4df7972591ba15629fcf&oid=d8e6eec26bf544f084bad49a7fa2eac5&exp=1505767774&rn=3533087049&tc=1&ct=a&sig=e622eb6e2b90ad8d2ba82a16b0fc7023cc9e086cb3dde5ce0847ac3d43aba84d&pbs=0930c744592849efab20b8155b32bb87" -c:a copy -c:v copy "OUTPUT2.ts"


ffmpeg -i "https://content-ausc1.uplynk.com/6c68ec3ffc3c4df7972591ba15629fcf/j.m3u8?cid=6c68ec3ffc3c4df7972591ba15629fcf&oid=d8e6eec26bf544f084bad49a7fa2eac5&exp=1505770655&rn=1539958891&tc=1&ct=a&sig=66a38b69a01a32ecfc3d02f5305c62aa9df37404fe573726e6c42fb82986595e&pbs=93612c07ddd945b0b8b8ca314df9a02c" -c copy -absf aac_adtstoasc "name of file.mp4"

This works, I guess since it is reaching back to the "crypto server"/"key file" this is why the process is so slow? The process if you strictly use "Video DownloadHelper" takes maybe 10 minutes. The full process above takes 20-30 minutes. I wish i knew how Video DownloadHelper did the process so much faster.

Example of what ffmpeg spits out every few seconds:

[hls,applehttp @ 0x7f86fd800000] Opening 'crypto+https://stgec-ausw.uplynk.com/80C078/ausw/slices/6c6/d8e6eec26bf544f084bad49a7fa2eac5/6c68ec3ffc3c4df7972591ba15629fcf/J00000089.ts?pbs=93612c07ddd945b0b8b8ca314df9a02c&si=0' for reading

If anyone can help, point to a different way, or anything please respond or even PM me. Thank you!

[lottery IcTv]

youtube-dl.exe "https://content-ause1.uplynk.com/6c68ec3ffc3c4df7972591ba15629fcf/k.m3u8?pbs=a8a398f1675e40b198216a1aefe8b0d8"

1080p Video Check If Any Issues
http://www84.zippyshare.com/v/5tCZcVgy/file.html
http://www84.zippyshare.com/v/LPtIKAl0/file.html
http://www84.zippyshare.com/v/sBnwfnUd/file.html
http://www84.zippyshare.com/v/IQNool1L/file.html
http://www84.zippyshare.com/v/zUczCOCt/file.html
http://www84.zippyshare.com/v/SOsUtWuJ/file.html
http://www84.zippyshare.com/v/89XIWFtK/file.html
http://www84.zippyshare.com/v/9q16ynqm/file.html
http://www84.zippyshare.com/v/MwDWAtw4/file.html

[ehurdler]

lottery IcTv thanks. This process which I thought I had tried before, and didnt work, well did this time. Unfortunately This no longer works. Youtube-dl or my long process i listed above. For example, the new episode of Brooklyn Nine-Nine https://www.fox.com/watch/5f5afb41c6...189e08c56e912/ When you simply use youtube-dl and one of the quality m3u8 files, it downloads everything but adds an advertisement in the beginning of the downloaded video. The advertisement also cuts off the beginning 15 seconds of the episode.

I will say, I am using a mac and when I use youtube-dl on certain sites I have to use the " --prefer-ffmpeg" option or it will error out. So what I tried is

youtube-dl https://content-ausw3.uplynk.com/e9ccef09f5234545aa67eacf0a1aadeb/j.m3u8?pbs=7d7129d55e794343be80df1da9944c1e --prefer-ffmpeg

I do get the video downloaded, but as I stated gets advertisement stuck to the beginning. Is there any way around this?

[Yelloworanges]

The youtube-dl contributors are probably working on it.

The steps described in the next to last post here:
https://github.com/rg3/youtube-dl/issues/14147

Seem to work.

1. Download/view theplatform request.
2. Look for the PlayURL in the request.
3. Edit the PlayURL and download with streamlink/livestreamer.

[ehurdler]

Can you explain further? I tried the process described in the github link you sent. I parsed the url, as what I thought was right. I was able to use livestreamer but I still get the commercials. Can you give detailed instructions if you got this working? Thank you.

[Yelloworanges]

Originally Posted by ehurdler

... For example, the new episode of Brooklyn Nine-Nine https://www.fox.com/watch/5f5afb41c6...189e08c56e912/ When you simply use youtube-dl and one of the quality m3u8 files, it downloads everything but adds an advertisement in the beginning of the downloaded video.

I just checked the above video using the github method. No ad at the beginning or anywhere else.

Originally Posted by ehurdler

Can you explain further? I tried the process described in the github link you sent. I parsed the url, as what I thought was right. I was able to use livestreamer but I still get the commercials. Can you give detailed instructions if you got this working? Thank you.

I really don't know how to make it any clearer. It is 3 steps.

Use the very long theplatform request made by the browser (no editing this or using an alternatively formed request).

Once you have this data, at the very top there is a very long uplink preplay json. Load that (again, no edits).

Once you have that, the PlayURL is at the very bottom of the preplay json. The PlayURL is the only item that needs to be edited. The pbs value at this particular location is important in determining what is sent in the m3u8. The m3u8 here is a parent m3u8 and not a quality m3u8.

If you are following these steps without doing any shortcuts or kludges, then I don't know what else to suggest.

[confused.com]

Originally Posted by ehurdler

I do get the video downloaded, but as I stated gets advertisement stuck to the beginning. Is there any way around this?

ffmpeg -ss 5 -i inputfile outputfile

discards the first five seconds of input.

If your input file was 60 seconds long, the output file will be 55 seconds long.

https://superuser.com/questions/5380...fset-in-ffmpeg

https://trac.ffmpeg.org/wiki/Seeking

[ehurdler]

OK, Yelloworanges. I got it working now. I swear I did what that github post said and I got the file with commercials. I tried it again this morning and I got the file without commercials. To be fair, I just didn't fully understand your instructions. At first I was trying to find "theplatform" thru Tools-Web Developer-Network. This does get what you search for, but it is missing I guess some of the header data. But using the Web Console, now I see how to get what is needed. I can also find it thru AdBlock Plus. Just have to toggle that on and off from blocking on fox.com or it won't load the videos for me initially.

I did try in a test also if to take and parse the link that hls stream detector finds in firefox. That didn't work last night when I tried it, but it did work this morning. So, Thank you for the instructions. I guess I needed a bit more detail. I appreciate the help!

[Yelloworanges]

Originally Posted by ehurdler

OK, Yelloworanges. I got it working now. ... I tried it again this morning and I got the file without commercials. To be fair, I just didn't fully understand your instructions.

I guess I needed a bit more detail.

Now that you can do it, how would you have explained it better? This might help others so they don't have to deal with my lousy unclear instructions.

[cabulo]

Solution that cannot be more straightforward, simply cut the commercial off the saved video with ffmpeg.