be-at.tv changed their internal urls - how to download?

Re: be-at.tv changed their internal urls - how to download?

 

Swiss File Knife is an ideal choice.

Download sfk169.exe from sourceforge.

Description of the feature that would do the job.

Very crude example of batch file (has little error checking) that uses sfk169.exe to strip the bytes and copy them with the first file to joined.flv.

joinflv.bat:

The batch file is hard coded to start with 0001.flv, so you need only to pass the last file number (without the file extension) as a command line parameter;

-----
Edit:

Check your source files carefully before using this batch file on any non-AVC encoded session, or any session other than:

as the session may be encoded with vp6, or have a different offset for the start of the 'AVC sequence header', as mentioned by KSV in the post below.

In all cases, this binary file copy method will work, but for some (possibly many) sessions, the offset may need to be changed from 32, to quite often 13, but possibly other values as well..

However, ffmpeg with work with many sessions. All my attempts to maintain sync, with this session in particular, failed with the concatenate feature of ffmpeg, although this is probably not a problem with ffmpeg itself. Note that due to the source material in these files, it can be difficult to check sync. I checked it by comparison of the waveform segment of the audio and the corresponding video frame, as well as the overall time length of the video.

Re: be-at.tv changed their internal urls - how to download?

 

Stripping first 32 bytes isn't gonna work reliably with different files. you are assuming that first flv packet is 'AVC sequence header' and it's of same length for different files which isn't true. a better way to join those files is explained below:

1. create a new folder (BeatConvert) and copy BeatConvert.php, FlvFixer.php, FFMpeg.exe to this folder.

2. now create a new bat file (BeatConvert.bat) with following content:
3. now copy your beat files (0000.beat....nnnn.beat) to the same folder and run the bat file.

if everything goes as expected a new file (Final.mkv) will be created which should be in sync.

Re: be-at.tv changed their internal urls - how to download?

 

Yes, I know. For the v1 .beat files, or some others encoded in vp6, it is 13 bytes. But whatever it has been, it has always been consistent within a session for all .beat files 0001+ through the end of the session. In any case, this parameter is easily edited as needed.

I don't know what to tell you, except, as I mentioned earlier in this thread, and in the other thread, my favored method was the concatenate feature of ffmpeg. But it bombed. And of course I tried FlvFixer.php, both on the segmented .flv files, or the joined, or both, and everything it between. I also just tried direct stream copy (individually) of the segmented .flv files with ffmpeg prior to concatenation in order to "fix" the .flv from the way in which it is time coded and yet still maintain sync.

Now, for the session that has most recently been in discussion in this thread;

Which has corresponding .beat files at;

I just tried your batch file using the raw source .beat files.
The resulting files it produced:

Final_fixed.flv Time length: 01:35:34
Final.mkv Time length: 01:35:34

Time length of video when played in flashplayer: 01:32:28

Time length of video when joined by binary file copy method I mentioned in my previous post: 01:32:28 (it could show up as 01:32:27 on some players)

If you have any ideas, I'm all ears.

But I do agree to caution anyone before using the batch file that I just posted to check the source .flv files after processing with BeatConvert.php.

Re: be-at.tv changed their internal urls - how to download?

 

can you post the debug log of FlvFixer.php?

Re: be-at.tv changed their internal urls - how to download?

 

Sure, thanks. The log seems oddly formatted? But here is what spit out from the command line you specified.
Edit: There was an error in the first copy of the above log.
Updated Debug.zip w/ Debug1a.zip.

It could be that something is off with their encode of just this particular session.
For comparison, here is the debug log from FlvFixer.php for the .flv file joined by binary file copy rather than ffmpeg.

Re: be-at.tv changed their internal urls - how to download?

 

i have analyzed your logs. strangely the file concatenated by ffmpeg is missing most of audio video data contained in 0000.flv but when i try it on my side everything seems ok. are you sure you are using the latest version of ffmpeg?

Re: be-at.tv changed their internal urls - how to download?

 

hmm. that's kinda odd. I'm at a loss.

I had tried several versions of ffmpeg along the way while working with this particular set of files from this session.

I just used the newest version:
ffmpeg version N-60240-ge1cb6dc
built on Jan 29 2014 22:01:42 with gcc 4.8.2. (GCC)

Are you using the same version of ffmpeg referenced above? And it produced a ffmpeg concatenated file with a length of 01:32:28 (or 01:32:27 on some players)?

For anyone else following this thread, use the ffmpeg method shown in ksv's example. If the time length of the produced file matches the web player (or is -1 second off), it is very likely synced properly. In all cases, this binary file copy method will work, but for some (possibly many) sessions, the offset may need to be changed from 32, to quite often 13, but possibly other values as well.

Re: be-at.tv changed their internal urls - how to download?

 

Final_fixed.flv Time length: 01:35:34
Final.mkv Time length: 01:35:34

same here with using KSVs batch. :-(

Re: be-at.tv changed their internal urls - how to download?

 

Well, could you test my batch too? It only takes a few seconds. Just copy the batch file and sfk169.exe into a folder with 0000.flv - 0462.flv (the .flv files processed already with BeatConvert.php, not the .beat files) from the timewarp-2012/marcel-fengler session.

Ffmpeg will quite often work with many sessions, but I have found about a dozen sessions that it has problems with, although this is probably not a problem with ffmpeg itself. In all cases, the binary file copy method will work, but for some (possibly many) sessions, the offset may need to be changed from 32, to quite often 13, but possibly other values as well. When you do the binary file copy method, you have a copy of the server file as they muxed it - as there is no remuxing involved.

Re: be-at.tv changed their internal urls - how to download?

 

@blimey: joined.flv is 01:32:27, thanks! :-)

Anyone knows (if we can't find a solution that works for all types), how I can easily see if I should use your batch, KSVs batch, or even have to alter your batch to a different size of byte to strip?

Re: be-at.tv changed their internal urls - how to download?

 

Guys, we're so close! Just need to know how I can find out if and how much bytes I'd have to strip! :-)

Re: be-at.tv changed their internal urls - how to download?

 

@blimey
i didn't noticed your corrected debug log you posted in your previous post. you should have mentioned it in new post. anyway i have analyzed your new log and it's obvious why total time of joined file is increased. it's due to wrong interleaving of audio video packets at the point of separation.

End of first flv:
Start of second flv:
due to non monotonically increasing timestamps ffmpeg's concat filter rebases second file's timestamp starting from 12376. this results in increased total length of file.

Solution:
i have modified my BeatConvert script to generate single flv file instead of one file per fragment so we don't have to use concat filter at all avoiding this whole mess. use the updated script with following batch file and report. it should create flv file with proper sync and length.

BeatConvert.bat:

Re: be-at.tv changed their internal urls - how to download?

 

thanks KSV

i would be glad if you option your PM again switch on

Re: be-at.tv changed their internal urls - how to download?

 

That's great KSV, thanks! That point in the log did stand out to me, but I wasn't sure what it meant.

Anyway, thanks for the updated script. This project is finally finished, for now at least.

Re: be-at.tv changed their internal urls - how to download?

 

I tested the updated script on two AVC sessions, and all is correct.

Final.flv
Final_fixed.flv
Final.mkv

each have the correct duration and are in sync. I will test it on a vp6 session later, but it looks like all is good with the updated script!

Re: be-at.tv changed their internal urls - how to download?

 

I will test it this weekend, but I think this will be the final solution.

KSV: I can't thank you enough for coding this. I have been waiting so long for a solution for my favourite video site.

blimey: Also thanks for your expertise and energy in this.

You both are my heroes! I will give you an info when I tested this. :-)

Re: be-at.tv changed their internal urls - how to download?

 

KSV,blimey
respect:cool:

Re: be-at.tv changed their internal urls - how to download?

 

I tested a few sessions, and all are in sync! Again: Thank you so much!! :-)

Re: be-at.tv changed their internal urls - how to download?

 

i must also say: thank you very much, all of you, guys, for this. it's a very big help for me and my friends, we like to record and share be-at.tv, and had to do it while streaming it lately.. i really appreciate this, cause it's already saving me tons of time.. =)

i am really after the audio though, video - yes, some of the sessions, maybe every 5th, no matter, and audio was kinda left out of your discussion...

i do want to ask something though... i feel kinda stoopid and cannot figure it out, where do we find all 3 of the IDs - the Recording ID (all djs/mixes as one club night), Session ID (one dj/mix), and Stream ID (actual 128/192/1000/2500/etc).. there's also 4th ID, its irrelevant now, i think..

as was discussed, there are currently 2 ways they serve their streams (and i am using 2 most recent sessions):

1. as in here ..be-at.tv/brands/klokgebouw/awakenings-eindhoven-2014-area-y/makcim-b2b-el-p this one yields pieces like this cdn.be-at.tv/003789/004433-0128/005808-1000/0003.beat that satisfy the original example where it allows one to download the audio stream with dtmpdump, and i do that, substitute my own session id and stream id, and i get 128kbps version (in this case) of the audio stream.

2. as in here ..be-at.tv/brands/klokgebouw/awakenings-eindhoven-2014-area-x/ - this one produces different pieces, that are accessed like this ..cdn.be-at.tv/002288/192/1250/0097.beat - and in this case, whatever i do, the most i can find is the session ID - something like 3801 if i remember right, but i cannot find stream ID. i am trying all possible ways. i employed like 6 different url sniffers, nothing.. i did look into .program file they served, and it had the Session ID in there, but not the stream ID. and so i am rendered helpless facing this task: finding the 2 ids to download the stream smoothly.

i did though download all the video and audio in some other case, where i didn't want video, btw, ..cdn.be-at.tv/002288/192/0097.beat - produces just the audio, converted from flv to mp3 and was happy, still saved tons of time.

i just wanna know, how did you guys find those streams and their IDs up above in the posts?

thanks, will be eagerly checking back here... =)

Re: be-at.tv changed their internal urls - how to download?

 

@tribalmixes: Sorry, but sharing their content is a violation of copyright. This community is only for our personal purposes.

Be-At.tv is streaming so much great content, and they are deserving every click!!

Re: be-at.tv changed their internal urls - how to download?

 

That seems like a good solution for just the audio, 002288/192/ ranges from 0095-1074 (.beat), and seems to include the audio (and only the audio) in acc format (flv) for half of the four sessions of Awakenings Eindhoven 2014 Area X. The other half is in 2290.

I'm not sure if they are currently actively using the non-beat format sources, although for some reason they are still present. I don't remember where the Audio00xxxx and Video00xxxx addresses are located, I'd have to check again. I think I uncovered them when experimenting with different user agent settings for the browser.

Re: be-at.tv changed their internal urls - how to download?

 

you wanna be a dick about it - go ahead, my pleasure. but please, make sure erase this thread, cause it does allow assholes like me to find out how to get their music and share it. you wanna be the only one who knows how to rip be-at.tv? well i must surprise you, there are a few more people now, including me.

you wanna talk copyright? their TOS perfectly state that copying, recording, and whatsoever, is as prohibited as sharing their content. standard TOS, i must note. they know how to deal with their stuff being copied and shared and do it sometimes. who are you? do i see there some kinda cape? you like professor hindsight? the superhero? congratulations! you are the winner.

Re: be-at.tv changed their internal urls - how to download?

 

well yeah, i noticed when video screen is minimized in that setting they stop doing video, thus it's the audio only in there, but to be honest, rtmpdump works way more impressive. in my opinion..

i'll try to play with those myself, thanks alot!! for the help and info and all.

Re: be-at.tv changed their internal urls - how to download?

 

There are no old style links to any of the four sessions in the Awakenings Eindhoven 2014 Area X event.

All of the last dozen or so events in the "Recently Recorded' list have old style links, with the exception of the Awakenings Eindhoven 2014 Area X event. I have no idea why that is the only event that lacks them. Perhaps because it was just recorded four days ago?

It is the only event/session I've found that doesn't have old style links in addition to the new format. If there is another one, provide the details.

Re: be-at.tv changed their internal urls - how to download?

 

I got a new problem:

When decoding the following session:

the parts in the middle aren't decoded properly, resulting in "green screen". However, the audio is ok.

Anyone knows what could cause the problem?

Re: be-at.tv changed their internal urls - how to download?

 

I took a quick look. Something goes wrong at all of the joints. However, the fragments are joined properly by the script.

Each of any of the fragments play ok individually with any player.

As for the joined files;

Final.flv (the decoded/joined file prior to processing with FlvFixer.php) will play without issue with the flv player that comes with getflv. But playback fails at the joints with most other players.

Final_fixed.flv (post FlvFixer.php processing) will no longer play with the getflv player.

ffmpeg throws numerous errors when copying the contents into the mkv container;

Non-monotonous DTS
missing picture in access unit with size 40
reference count overflow
illegal memory management control operation

I'm not sure what is the problem. The easiest solution is to transcode the individual frgaments and join them as you transcode. I tested that and it worked ok.

You could download the video from the old system (Video00xxxx links), that joins ok, but since it is vp6 you would probably want to transcode it anyway.

Re: be-at.tv changed their internal urls - how to download?

 

My video plays without distortions, but Final_fixed.flv = 1KB (Final.flv > 2GB) :(

Re: be-at.tv changed their internal urls - how to download?

 

We have different source files from each other. The source I was speaking about is the 960x540 resolution at /1250. Yours is the 1024x576 resolution at the /2500. So, I suppose, be sure to download the higher resolution source files for this session.

Final_fixed.flv did have the full size for the lower resolution source since the input file size is only 1.86 GB. But for the higher resolution source, the input file size, as you say, is > 2 GB, and this seems to be beyond the limit for the FlvFixer.php script.

Re: be-at.tv changed their internal urls - how to download?

 

This particular session (http://www.be-at.tv/brands/dc10/para...13/jamie-jones) have 0000.beat file which contains different AVC sequence header than rest of other beat files. it seems they restarted video encoding with different parameters after first few seconds. this causes problems with most players because they expect only one decoder configuration packet per stream. you can leave out 0000.beat and try to join the rest. it should work properly though i have tested it only on first few fragments.

for the 2 GB problem it's a limitation of php which uses signed 32 bit file pointers limiting the file size to 2 GB. if you are using a 64 bit operating system (greater than xp) you can download 64 bit php and try with that.

Re: be-at.tv changed their internal urls - how to download?

 

Alright, thanks again!

Re: be-at.tv changed their internal urls - how to download?

 

The /2500 was never shown here. Is there a button to switch to "hd" which I can't find?

Re: be-at.tv changed their internal urls - how to download?

 

Found it. It seems like switching to fullscreen will load the next beat-file with the better quality. :-)

Re: be-at.tv changed their internal urls - how to download?

 

kvs or/and Blimey could you please help me with this:



?

It is saying one flv file is broken.. but how do i find out wich one?

Re: be-at.tv changed their internal urls - how to download?

 

Found it! Total file size is more than 2 gb.. I'm gonna do this in 2 parts and find a program to join the mkv's :-)

Re: be-at.tv changed their internal urls - how to download?

 

have you guys managed to find a way to grab the audio successfully yet? i cant for the life of me do it - im trying to get the mp3 of this link http://www.be-at.tv/FcwFAA can anyone help me? thanks !

Re: be-at.tv changed their internal urls - how to download?

 

The old system has been entirely disabled. So, the old rtmp server that had mp3 audio is gone.

Audio-only can still be downloaded in beat format, which is 192k aac. For the content you specify, you can get audio-only (no video) by downloading these 1904 beat fragments;
After downloading, process them with the BeatConvert.php script.

Re: be-at.tv changed their internal urls - how to download?

 

thanks for the reply dude , im pretty newbie with scripts etc - i downloaded the two files you linked but not sure what to do with them next, could you help? thanks man

Re: be-at.tv changed their internal urls - how to download?

 

Hmm :( Seems noone can help... Thanks anyway

Re: be-at.tv changed their internal urls - how to download?

 

Hello, new here and to how video stream recordings work in general honestly :p

I was wondering if anyone can help download this video to my macbook please: http://www.be-at.tv/brands/dalt-vila...l-englund-live


Thank you very much in advance!!!

Re: be-at.tv changed their internal urls - how to download?

 

hi, anyone can this thread translate in german ?

how can i find the .beat files ?

how can in convert the .beat files in mp4 etc. ?



http://www.be-at.tv/brands/nature-on...aturday/dj-dag