Audio/video stream recording forums (http://stream-recorder.com/forum/index.php)
-   Video stream recording (http://stream-recorder.com/forum/forumdisplay.php?f=4)
-   -  

How to use iView (ABC Australia) downloaders

(http://stream-recorder.com/forum/showthread.php?t=7039)

mce 11-02-2010 09:35 PM

Re: How to use iView (ABC Australia) downloaders


 
Am looking into it now.

Yansky 11-02-2010 10:52 PM

Re: How to use iView (ABC Australia) downloaders


 
Even though I'm subscribed to this thread I don't seem to be getting the updates emailed to me. :confused:
(i checked the spam folders in my gmail too)

Anyone else not getting updates?

mce 11-02-2010 11:12 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Yansky (Post 22710)
Anyone else not getting updates?

Sorry mate, no email thingy set up here.

Both iViewNapper and iViewNapperLite have been updated.

o Fix for new AUTH URL.

blood.eaglz 11-02-2010 11:17 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Yansky (Post 22710)
Even though I'm subscribed to this thread I don't seem to be getting the updates emailed to me. :confused:
(i checked the spam folders in my gmail too)

Anyone else not getting updates?

Same here, I know this place has a wierd subscription system in that you have to visit the thread after each notification whilst logged in, or it wont send future updates, yet AFAIK I did that ages ago, I check in today due to iviewnapper closing on fetching the latest QI, and there are 5/6 new pages of posts that I knew nothing about...

EDIT: humph, damn system decided to work again, just got notification of mce's post whilst posting :)

Stream Ripper 11-03-2010 12:14 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by blood.eaglz (Post 22713)
Same here, I know this place has a wierd subscription system in that you have to visit the thread after each notification whilst logged in, or it wont send future updates, yet AFAIK I did that ages ago, I check in today due to iviewnapper closing on fetching the latest QI, and there are 5/6 new pages of posts that I knew nothing about...

EDIT: humph, damn system decided to work again, just got notification of mce's post whilst posting :)

Thanks for the update. :)

johnm 11-03-2010 02:52 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by mce (Post 22712)
Sorry mate, no email thingy set up here.

Both iViewNapper and iViewNapperLite have been updated.

o Fix for new AUTH URL.

FYI
For some time "iViewNapperLite" keeps crashing on shows that are longer than an hour. Just over 300000kB. It varies every time. The error is "ERROR. RTMP_ReadPacket-Failed to read RTMP packet header
INFO Connection timed out, trying to resume."

"iViewNapper" also has an issue at about the same place but seems to restart successfully.

Thank you for all your good work.

Yansky 11-03-2010 03:13 AM

Re: How to use iView (ABC Australia) downloaders


 
I've updated iViewFox: https://addons.mozilla.org/en-US/fir...0505/versions/

BTW, has anyone compiled a version of rtmpdump that works on OSX 10.4? I meant to do this a while back but I forgot. :o

darian 11-03-2010 04:45 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by gxdata (Post 22706)
Wed 3Nov2010 - iViewNapper is crashing for me, any program, but the batch downloader is working for them.

Having same problem myself.

Anywho 11-03-2010 01:53 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by darian (Post 22729)
Having same problem myself.

See post #282 of this thread :D

johnm 11-03-2010 10:19 PM

Re: How to use iView (ABC Australia) downloaders


 
I posted the comments after updating to the latest version. It has been happening for sometime but not sure when it started thought.
Try downloading some of the long shows and you will see what I mean. There are only a few greater than 300MB at the moment. e.g. 2 x "Pirot"s.

Anywho 11-03-2010 10:47 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by johnm (Post 22744)
I posted the comments after updating to the latest version.

Who are you replying to? Please quote when you reply! :)

gxdata 11-04-2010 12:03 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by mce (Post 22712)
Both iViewNapper and iViewNapperLite have been updated.

o Fix for new AUTH URL.

Great! Again, thanks.

Anywho, your authentication was not affected - why? I'll have to compare the two.

gxdata 11-04-2010 02:25 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by johnm (Post 22744)
I posted the comments after updating to the latest version. It has been happening for sometime but not sure when it started thought.
Try downloading some of the long shows and you will see what I mean. There are only a few greater than 300MB at the moment. e.g. 2 x "Pirot"s.

To be clear: it is true that you're having this problem (halt, then resume) with iViewNapperLite v20101103, for videos that are bigger than about 300K?

tilt 11-04-2010 03:09 AM

Re: How to use iView (ABC Australia) downloaders


 
I would like to thank mce for the speedy effort. I have updated my variant of the batch downloader, which does 3 channels abc sbs plus7) using the show list and wanted list. If anyone else is interested in this pretty rough script, please say so.

gxdata 11-04-2010 03:58 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by tilt (Post 22753)
I would like to thank mce for the speedy effort. I have updated my variant of the batch downloader, which does 3 channels abc sbs plus7) using the show list and wanted list. If anyone else is interested in this pretty rough script, please say so.

Yes please - I'd like to have a look and a try.

gxdata 11-04-2010 08:43 AM

Re: How to use iView (ABC Australia) downloaders


 
Something drastic has happened - nothing can be found / downloaded, from 3hours30mins ago (see date/time on this message) for an hour or so.
Affected both the iVNapper and iVBatch, latest versions.
Now (another 2 hours later), the iVBatch is failing on The Drum (last 2 days), yet the iVNapper gets it OK. It's not making a lot of sense (no authent errors seen on iVBatch).

Anywho 11-04-2010 02:47 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by gxdata (Post 22758)
Something drastic has happened - nothing can be found / downloaded, from 3hours30mins ago (see date/time on this message) for an hour or so.
Affected both the iVNapper and iVBatch, latest versions.
Now (another 2 hours later), the iVBatch is failing on The Drum (last 2 days), yet the iVNapper gets it OK. It's not making a lot of sense (no authent errors seen on iVBatch).

Try again... iVB is working for me (@7:14AM-CDT). The Drum 2,3&4/11 d/l's fine here :confused:

Anywho 11-04-2010 03:00 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by tilt (Post 22753)
I have updated my variant of the batch downloader, which does 3 channels abc sbs plus7) using the show list and wanted list. If anyone else is interested in this pretty rough script, please say so.

ffff. fff. ffff. You ready? ..."so"...

We've shown you ours, now you show us yours! :D

gxdata 11-04-2010 05:02 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Anywho (Post 22767)
Try again... iVB is working for me (@7:14AM-CDT). The Drum 2,3&4/11 d/l's fine here :confused:

Moi, Je suis confus, trop!

Code:

Downloading "The Drum 04/11/10" ...
and saving the stream with the filename "The Drum 04-11-10.mp4"

Downloading from Hostworks server...

Retrieving Token...
Resuming previous download for "The Drum 04/11/10"...
RTMPDump v2.2d (r474)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
mp4:news/thedrum_101104 path not found. Retrying a different path...

Retrieving Token...
Resuming previous download for "The Drum 04/11/10"...
RTMPDump v2.2d (r474)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
news/thedrum_101104 path not found. Retrying a different path...

Retrieving Token...
Resuming previous download for "The Drum 04/11/10"...
RTMPDump v2.2d (r474)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
news/thedrum_101104.mp4 path not found. Retrying a different path...


The show "The Drum 04/11/10" was not found on the server.

This, Friday 5Nov2010 10:00AM Melbourne time.

Anywho 11-04-2010 05:34 PM

Re: How to use iView (ABC Australia) downloaders


 
1 Attachment(s)
Quote:

Originally Posted by gxdata (Post 22770)
Moi, Je suis confus, trop!

The show "The Drum 04/11/10" was not found on the server.[/code]

This, Friday 5Nov2010 10:00AM Melbourne time.

Hmmm... Have you deleted any 0-byte files that may have been created? I'm not getting any problem d/ling it - :confused:

The attached pic I just got 5 minutes ago!

gxdata 11-04-2010 05:52 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Anywho (Post 22772)
Hmmm... Have you deleted any 0-byte files that may have been created?

Ooops - yes, that was the problem. :o

Thinking about unattended (scheduled) DL for things I want to see, when I am remote from a TV or computer for 1 to 3 days, I wonder how to get over these periods of server or whatever problems that ABC/iView seems to encounter. I'll code something up, possibly next week, and share the workflow.

Anywho 11-04-2010 06:26 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by gxdata (Post 22774)
Thinking about unattended (scheduled) DL for things I want to see, when I am remote from a TV or computer for 1 to 3 days

Have you tried the Windows scheduler and the batch file d/ler, IView Later, I created?

Quote:

I wonder how to get over these periods of server or whatever problems that ABC/iView seems to encounter. I'll code something up, possibly next week, and share the workflow.
What do you mean by this? Like reacting to server dropout, non-existent files, etc?

tilt 11-05-2010 04:52 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Anywho (Post 22768)
ffff. fff. ffff. You ready? ..."so"...

We've shown you ours, now you show us yours! :D

Well Anywho you must have some retained air after that effort!

I have stuck the fetchtv.sh script at:

http://pastebin.com/HuSkwzm8

It is still riddled with debugging statements, but at least there is bash script code to handle the json, and plus7 and sbs. I run it under windows 7 or XP with cygwin, but could easily be modified to work with lynux

gxdata 11-05-2010 04:58 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by Anywho (Post 22776)
Have you tried the Windows scheduler and the batch file d/ler, IView Later, I created?
No, will do so.


What do you mean by this? Like reacting to server dropout, non-existent files, etc?

Exactly. When to retry, which programs have short residence times (eg, midday report, evening news), exception handling so that the app controlling rtmpdump doesn't die, reading authentication and other primary paths from a config file, putting json parsing into a DLL that can be swapped when the format changes, etc.

vipher 11-05-2010 07:19 AM

Re: How to use iView (ABC Australia) downloaders


 
I've updated the Linux script (now v6.8.1) and it's on the wiki, I think the same script should work with OSX just by changing the rtmpdump binary.

streamed 11-07-2010 12:09 AM

Re: How to use iView (ABC Australia) downloaders


 
Hi,

Had to register for this forum, this seems to be the place to ask questions about functionality for Iview grabbers/downloaders.

I would like to know if one of you can build some sort of QA tool to check/verify the downloads. Often I find myself watching a downloaded episode, when suddenly the download seems to be incomplete. End of video, bugger.

I have had incomplete downloads with Iviewnapper and also with IviewFox. They all use the same RTMPDump, so that seems to be the problem (and possibly also the solution?)

Would it be possible to have a little tool that scans a certain folder on your local PC, and verify the length of each of the media files with the length/size as stored/reported on the server? If the file is not on the server anymore, bad luck, but for the files that are still there, wouldn't that be very practical?

I wouldn't mind programming something like this myself, but I have zero knowledge of RTMPDump, so I would have to familiarise myself with the syntax etc.

I would think that if you already have the source to download the media files, it would't be too difficult to built this verification tool?

Or am I the only one with incomplete downloads?

gxdata 11-07-2010 01:02 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by streamed (Post 22802)
Hi,
I would like to know if one of you can build some sort of QA tool to check/verify the downloads. Often I find myself watching a downloaded episode, when suddenly the download seems to be incomplete. End of video, bugger.

I have had incomplete downloads with Iviewnapper and also with IviewFox. They all use the same RTMPDump, so that seems to be the problem (and possibly also the solution?)

Would it be possible to have a little tool that scans a certain folder on your local PC, and verify the length of each of the media files with the length/size as stored/reported on the server? If the file is not on the server anymore, bad luck, but for the files that are still there, wouldn't that be very practical?

(edited quote above)

With iViewNapper, there is an indication (SUCCESS) in a panel on the downloader form, which is some help.

About 6 months ago, I had the same thought as you, and asked of (someone - ? rtmpdump coder) if a checksum (CRC, MD5) could be ascertained for a file on the server, and checked against what you downloaded. I answered myself, that this would be a silly idea.

I'm not sure what metadata is available for the files that get streamed (ie, the files on the server that rtmpdump "downloads" to us) and how good that would be as an indication of the success, partial success, etc.

Others may have a more definitive answer for you.

streamed 11-07-2010 02:54 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by gxdata (Post 22811)

With iViewNapper, there is an indication (SUCCESS) in a panel on the downloader form, which is some help.

I have not used Iviewnapper lately, found the fact that every instance of the software can only do on edownload at the time quite complex and inconvenient. Am now using Iviewfox, ideal for multiple transfers as the frontend is just the Iview website, and not some program.

Quote:

Originally Posted by gxdata (Post 22811)
I'm not sure what metadata is available for the files that get streamed (ie, the files on the server that rtmpdump "downloads" to us) and how good that would be as an indication of the success, partial success, etc.

The length (size) of the file would be ok, to check that against the local file size.

Quote:

Originally Posted by gxdata (Post 22811)
Others may have a more definitive answer for you

We'll wait and see. Thanks for your reply.

gxdata 11-07-2010 04:52 AM

Re: How to use iView (ABC Australia) downloaders


 
for mce - see the Plus!7 thread on Whirlpool
p7Napper Exception - Air Traffic Investigation 6Nov2010

jameskinds 11-07-2010 05:31 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by vipher (Post 22785)
I've updated the Linux script (now v6.8.1) and it's on the wiki, I think the same script should work with OSX just by changing the rtmpdump binary.

Can you please repost this somewhere else as localhostr.com seems to be down.

Thanks in advance.

James

vipher 11-07-2010 08:07 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by jameskinds (Post 22822)
Can you please repost this somewhere else as localhostr.com seems to be down.

No problem, new link is on the wiki now.

jameskinds 11-07-2010 10:17 PM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by vipher (Post 22823)
No problem, new link is on the wiki now.

Thank you!! That's fantastic!! :)

Much appreciated.

blood.eaglz 11-08-2010 12:18 AM

Re: How to use iView (ABC Australia) downloaders


 
hey any chance of someone making a ten downloader (or if a working one already exists point it out to me), they migrated from roo to movideo, and now DownloadHelper gets bupkiss(ad video excepted) :(

mce 11-08-2010 02:14 AM

Re: How to use iView (ABC Australia) downloaders


 
p7Napper has been updated to version 20101108.

Please check the p7Napper website.

gxdata 11-09-2010 04:51 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by mce (Post 22834)
p7Napper has been updated to version 20101108.

Please check the p7Napper website.

Nice.
Pity 7 isn't quota-exempt like 2. There are a couple of OK programs there right now. Maybe I need to DL before 8am.

gxdata 11-09-2010 08:18 AM

Re: How to use iView (ABC Australia) downloaders


 
Interesting that evening news for 07-11-2010 is still available. The date/time now is 02:12 on 10Nov2010, AEDT.

Testing with "iViewLater" and editing the download_list.xml just now shows that the show for 7Nov is there, but 6Nov is not.

The catalog for ABC National Evening News says that only the show ABC National Evening News 09-11-10 is available.

Implication: possibility of getting some stuff that is still there, but was removed from the catalog - at least for a day or so. Or, it's an oversight, a once-off.

iviewthereforeiam 11-10-2010 12:17 AM

Re: How to use iView (ABC Australia) downloaders


 
I've downloaded the Firefox addon from Yansky and it seems to work fine, but I am wondering if the .mp4 files I download using that addon will be unmetered thru my ISP (iPrimus has unmetered iView downloads)

blood.eaglz 11-10-2010 01:48 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by iviewthereforeiam (Post 22877)
I've downloaded the Firefox addon from Yansky and it seems to work fine, but I am wondering if the .mp4 files I download using that addon will be unmetered thru my ISP (iPrimus has unmetered iView downloads)

yep, as the downloaders use the same 'path' as the site, it is indistinguishable to your ISP, and therefore it remains unmetered

gxdata 11-11-2010 07:34 AM

Re: How to use iView (ABC Australia) downloaders


 
Further to #315, I found that there are 7 or 8 days of "The Midday Report" that don't / have never appeared on the menus, but are downloadable. (3Nov to 11Nov)

Not that I'm interested in these per se, but it shows that what's left in place on the server isn't reflected in the edited JSON files describing what's available to watch.
Obviously, iView maintenance is a very manual process.

gxdata 11-11-2010 07:45 AM

Re: How to use iView (ABC Australia) downloaders


 
Quote:

Originally Posted by blood.eaglz (Post 22831)
hey any chance of someone making a ten downloader (or if a working one already exists point it out to me), they migrated from roo to movideo, and now downloadhelper gets bupkiss(ad video excepted) :(

StreamTransport will capture (for example) the Junior Masterchef stuff, with a bit of manual attention - but it's no faster than watching, afaict. You end up with a pile of .FLV files, since they have one week's/night's competition chopped up and prefaced by a sponsor's advert. (6 segments, as you can see from the Ten webpage).

I've never explored Ten, but for these (like JM) it seems feasible to use rtmpdump in the way that AdamMW recommended (now defunct websites for SBS and for Ten). I suspect that Ten is inconsistent compared with iView or even SBS, and that a downloader like iViewNapper or iView Batch Downloader would work on 10% to 30% of the programs/pages they present. Hence, too much maintenance to be worthwhile. (my guess only)


All times are GMT -6. The time now is 06:42 PM.