View Single Post
  #480  
Old 03-11-2011, 11:28 PM
gxdata gxdata is offline
Western Wombat
 
Join Date: Feb 2009
Posts: 332
gxdata is on a distinguished road
Default

Re: How to use iView (ABC Australia) downloaders


Quote:
Originally Posted by Yansky View Post
Are you able to set the request headers for you http request?

I think setting the accepted-encoding header "Accept-Encoding: gzip;q=0" will tell a server not to serve up the page using gzip.

http://www.w3.org/Protocols/rfc2616/...4.html#sec14.3
The blog post from Rick Strahl indicates the same, I think. But I am using WebClient() in .NET and don't see similar switches. The stream that I read is UTF-8 encoded, and gzip.exe / ungzip.exe doesn't recognize its format. I'm having a mental block atm.

err - the .Downloadfile method of the WebClient class is probably what I need. Yep - all go now.


They seem to be switching from GZ to text mode quite often, so the test for the "[{" characters will be needed at each new catalog fetch. I'm not sure what they are doing with the Series JSON files.
Reply With Quote