Re: How to use iView (ABC Australia) downloaders

gxdata · #**471** 03-11-2011, 03:55 AM

Quote:

Originally Posted by Anywho

LOL! Yes! They're flipping between text and compressed... Probably watching this tread (hi guys!)

Yes, the catalog (index) is in text, but the series json file just now failed (on the GUI). I guess it's not too hard to try both uncompressed and compressed, but a PITA.

gxdata · #**472** 03-11-2011, 03:59 AM

Weird - Mr Moon titles have changed since a few days ago, and its latest show is downloading (I just luv the kids shows).

But ['The Midday Report 11-03-11'] doesn't work for me. [EDIT - OK now]

Seems they are doing a wholesale edit of show titles, at least for the kids / ABC3 stuff.

[EDIT - I think a lot is going on - the Midday Report glitch was probably a normal "show-missing, back in 5 minutes" situation]

Anywho · #**473** 03-11-2011, 04:09 AM

Quote:

Originally Posted by gxdata

Yes, the catalog (index) is in text, but the series json file just now failed (on the GUI). I guess it's not too hard to try both uncompressed and compressed, but a PITA.

I've just compiled a standalone gzipper that seems to work. It's the one from the ZLIB library (minigzip-static.exe if anyone wants to have a go at it). It compiles to about 61KB... I'm just "trimming the fat" off it at the moment - we don't need compression.

gxdata · #**474** 03-11-2011, 04:20 AM

Quote:

Originally Posted by Anywho

I've just compiled a standalone gzipper that seems to work. It's the one from the ZLIB library (minigzip-static.exe if anyone wants to have a go at it). It compiles to about 61KB... I'm just "trimming the fat" off it at the moment - we don't need compression.

It should be possible (with .NET, otherwise) to work on the stream rather than save to disk, but I'm puzzled that my save of the stream is not convertible with gzip.exe, whereas if I point the online test tool (see post above) to the URL, it gets & unravels the json from [{ to }].

mce ?

gxdata · #**475** 03-11-2011, 04:24 AM

There's a lot of house-keeping going on at iView - Lateline from 1 and 2 March have just appeared. And the RSS feeds were screwed up earlier in the day, fixed in the last 30-60 minutes.

And Anywho is right - they are switching the gzip of the catalog / index on & off - I wonder how the iView Player is managing? It's an interesting little problem to code around, from our end!

Anywho · #**476** 03-11-2011, 05:12 AM

Quote:

Originally Posted by gxdata

And Anywho is right - they are switching the gzip of the catalog / index on & off - I wonder how the iView Player is managing? It's an interesting little problem to code around, from our end!

Nah... Just look for the opening "[{" in the first two (non-space) characters for the json text file.

(If anyone wants the ungzip (Windows) executable, you can grab it from here

)

mce · #**477** 03-11-2011, 04:22 PM

Just saw this thread.

Yep iVN fails at get catalog but iVNL seems to work 100% OK (at the moment).

gxdata · #**478** 03-11-2011, 10:33 PM

Sitting here coding, suddenly the catalog is back to text-based JSON!
12/03/2011_15:30 AEDT (UTC-11:00)

Yansky · #**479** 03-12-2011, 12:05 AM

Quote:

Originally Posted by gxdata

Sitting here coding, suddenly the catalog is back to text-based JSON!
12/03/2011_15:30 AEDT (UTC-11:00)

Are you able to set the request headers for you http request?

I think setting the accepted-encoding header "Accept-Encoding: gzip;q=0" will tell a server not to serve up the page using gzip.

http://www.w3.org/Protocols/rfc2616/...4.html#sec14.3

gxdata · #**480** 03-12-2011, 12:28 AM

Quote:

Originally Posted by Yansky

Are you able to set the request headers for you http request?

I think setting the accepted-encoding header "Accept-Encoding: gzip;q=0" will tell a server not to serve up the page using gzip.

http://www.w3.org/Protocols/rfc2616/...4.html#sec14.3

The blog post from Rick Strahl indicates the same, I think. But I am using WebClient() in .NET and don't see similar switches. The stream that I read is UTF-8 encoded, and gzip.exe / ungzip.exe doesn't recognize its format. I'm having a mental block atm.

err - the .Downloadfile method of the WebClient class is probably what I need. Yep - all go now.

They seem to be switching from GZ to text mode quite often, so the test for the "[{" characters will be needed at each new catalog fetch. I'm not sure what they are doing with the Series JSON files.