webcast.berkeley.edu: Capture Real video streams (protected by DRM?)

[moscontheroad]

Hello, I've been trying to find a way to download/record some of the classes from

http://webcast.berkeley.edu/course_details.php?seriesid=1906978259

I have noticed that not all of the courses are protected and in some cases just a few classes are, however there is no use in downloading an incomplete course, I'd be very grateful if anyone could help out! these are definitely in Real Video

There are many streaming video recorders that support Real video .RM streams broadcasted through the RTSP protocol.

I used to download video lectures (webcasts) from Berkeley.edu. In my tests NetXFer was the best software for the task. It is pretty easy to download multiple real video streams from webcast.berkeley.edu:

1. Open Internet Explorer and navigate to a web-page with the course details.
2. Right click on the web-page and select "Download all by NetXFer".
3. In the "Select URLs" dialog unselect everything, sort by the URL, select all URLs that contain "type=real" (click on the first one and then use holding shift press down arrow several times)
4. Click OK.

NetXfer can postprocess .RM metafiles and download rtsp:// .rm video streams automatically.

p.s. Although in my tests NetXfer was the best program for Berkeley webcasts, it wasn't perfect. Sometimes I had to re-download video lectures. The bummer is that NetXfer didn't show any error, so I had to check the lengths of all lectures manually.

p.p.s. I have never seen any DRM protected lectures at webcast.berkeley.edu. Please let me know if you find one.

[crs]

Jaksta is another choice.

Originally Posted by crs

Jaksta is another choice.

The reason I can't recommend WM Recorder, Jaksta, Replay Media Catcher, Replay AV for webcast.berkeley.edu is because these programs do require you to play all video lectures that you want to download. Even if you need to download 15-20 lectures it will take lots of time. Software with browser integration that allows to download all the files linked from a single web-page is much more convenient here.

[crs]

I only tested Jaksta on this site, so cant speak for the others. You just need to click on each lecture you want to start downloading in your browser so it picks it up. No need to continue playing them in your browser. I had 7 going at the same time.

Originally Posted by crs

I only tested Jaksta on this site, so cant speak for the others.

I was recommending software for webcast.berkeley.edu only.

Originally Posted by crs

You just need to click on each lecture you want to start downloading in your browser so it picks it up. No need to continue playing them in your browser. I had 7 going at the same time.

That's exactly the problem I was talking about. Instead of clicking onto the "Download all" button and choosing all media files easily, I need to play every lecture so that the URL is detected by the stream ripper/recorder software like Jaksta.

[any ANONYMOUS forum user]

Hi there,

I have tried using NetTransport for downloading berkeley webcast videos as you had suggested

The course link is

http://webcast.berkeley.edu/course_details.php?seriesid=1906978357

I right clicked on the page and selected "Download all files using NetXfer" and then chose only files of type=real but the initial files (.rm) get downloaded which are 88kb whereas the rtsp links are not getting downloaded.

I will paste the log in NetXfer for both the cases

For the smaller .rm file (which has the rtsp link in it)

6/15/2009 12:16:15 AM.953 Connecting proxy to iiscproxy1.serc.iisc.ernet.in:3128
6/15/2009 12:16:15 AM.968 Connecting proxy to 10.16.40.12:3128
6/15/2009 12:16:15 AM.984 Connected.
6/15/2009 12:16:15 AM.984 GET http://webcast.berkeley.edu/stream.p...ebcastid=16672 HTTP/1.1
6/15/2009 12:16:15 AM.984 Accept: */*
6/15/2009 12:16:16 AM.000 Referer: http://webcast.berkeley.edu/course_d...sid=1906978357
6/15/2009 12:16:16 AM.000 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)
6/15/2009 12:16:16 AM.000 Host: webcast.berkeley.edu
6/15/2009 12:16:16 AM.000 Proxy-Connection: Keep-Alive
6/15/2009 12:16:16 AM.000 Pragma: no-cache
6/15/2009 12:16:16 AM.000 Cache-Control: no-cache
6/15/2009 12:16:22 AM.250 HTTP/1.0 200 OK
6/15/2009 12:16:22 AM.250 Date: Sun, 14 Jun 2009 18:49:02 GMT
6/15/2009 12:16:22 AM.250 Server: Apache
6/15/2009 12:16:22 AM.250 X-Powered-By: PHP/5.1.6
6/15/2009 12:16:22 AM.250 Content-Disposition: attachment; filename="16672.rm"
6/15/2009 12:16:22 AM.265 Cache-Control: max-age=300
6/15/2009 12:16:22 AM.265 Expires: Sun, 14 Jun 2009 18:54:02 GMT
6/15/2009 12:16:22 AM.265 Content-Length: 88
6/15/2009 12:16:22 AM.265 Content-Type: audio/x-pn-realaudio
6/15/2009 12:16:22 AM.265 X-Cache: MISS from s2.serc.iisc.ernet.in
6/15/2009 12:16:22 AM.265 Via: 1.0 s2.serc.iisc.ernet.in (squid/3.0.STABLE8)
6/15/2009 12:16:22 AM.265 Proxy-Connection: close
6/15/2009 12:16:22 AM.265 Start receiving data.
6/15/2009 12:16:22 AM.265 Finished at 88

And for the RTSP file

6/15/2009 12:16:16 AM.937 Connecting proxy to 10.16.40.12:3128
6/15/2009 12:16:16 AM.953 Connected.
6/15/2009 12:16:16 AM.953 OPTIONS rtsp://169.229.131.16:554 RTSP/1.0
6/15/2009 12:16:16 AM.953 CSeq: 1
6/15/2009 12:16:16 AM.953 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
6/15/2009 12:16:16 AM.953 ClientChallenge: ee6153ed92c0e0301b2de92af0056644
6/15/2009 12:16:16 AM.953 PlayerStarttime: [15/06/2009:00:16:16 05:30]
6/15/2009 12:16:16 AM.953 CompanyID: 02KLghnr8RfHaCqcUoL3Kw==
6/15/2009 12:16:16 AM.953 Pragma: initiate-session
6/15/2009 12:16:16 AM.968 HTTP/1.0 403 Forbidden
6/15/2009 12:16:16 AM.968 Server: squid/3.0.STABLE8
6/15/2009 12:16:16 AM.968 Mime-Version: 1.0
6/15/2009 12:16:16 AM.968 Date: Sun, 14 Jun 2009 18:52:02 GMT
6/15/2009 12:16:16 AM.968 Content-Type: text/html
6/15/2009 12:16:16 AM.968 Content-Length: 1395
6/15/2009 12:16:16 AM.968 Expires: Sun, 14 Jun 2009 18:52:02 GMT
6/15/2009 12:16:16 AM.968 X-Squid-Error: ERR_ACCESS_DENIED 0
6/15/2009 12:16:16 AM.968 X-Cache: MISS from s2.serc.iisc.ernet.in
6/15/2009 12:16:16 AM.968 Via: 1.0 s2.serc.iisc.ernet.in (squid/3.0.STABLE8)
6/15/2009 12:16:16 AM.968 Proxy-Connection: close
6/15/2009 12:16:16 AM.968 HTTP server error.

If you notice there is an X-squid-error
Plz help me out

I've tried to download these video streams with NetTransport 2.84a. I do have problems with Net Transport here, it doesn't postprocess .RM metafils correctly. I apologize for that. I will report this bug to the developer.

However I don't have any problem downloading RTSP .RM files:

15.06.2009 11:34:50.329 Connecting to 169.229.131.16:554
15.06.2009 11:34:50.564 Connected.
15.06.2009 11:34:50.610 OPTIONS rtsp://169.229.131.16:554 RTSP/1.0
15.06.2009 11:34:50.610 CSeq: 1
15.06.2009 11:34:50.610 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
15.06.2009 11:34:50.610 ClientChallenge: fc7c9714500e1aaf167ac216a9b3e33e
15.06.2009 11:34:50.610 PlayerStarttime: [15/06/2009:11:34:50 03:00]
15.06.2009 11:34:50.610 CompanyID: bPrkxqUQFqdtb+RfkRBqWA==
15.06.2009 11:34:50.610 Pragma: initiate-session
15.06.2009 11:34:50.860 RTSP/1.0 200 OK
15.06.2009 11:34:50.860 CSeq: 1
15.06.2009 11:34:50.860 Date: Mon, 15 Jun 2009 07:34:51 GMT
15.06.2009 11:34:50.860 Session: 438785-1
15.06.2009 11:34:50.860 Server: Helix Mobile Server Version 11.1.8.4940 (sunos-5.8-sparc-server) (RealServer compatible)
15.06.2009 11:34:50.860 Public: OPTIONS, DESCRIBE, ANNOUNCE, PLAY, PAUSE, SETUP, GET_PARAMETER, SET_PARAMETER, TEARDOWN
15.06.2009 11:34:50.860 TurboPlay: 1
15.06.2009 11:34:50.876 RealChallenge1: 8911d92234e68e2a4567c821c31ebb45
15.06.2009 11:34:50.876 StatsMask: 7
15.06.2009 11:34:50.892 DESCRIBE rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39 RTSP/1.0
15.06.2009 11:34:50.892 CSeq: 2
15.06.2009 11:34:50.892 Accept: application/sdp
15.06.2009 11:34:50.892 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
15.06.2009 11:34:50.892 Session: 438785-1
15.06.2009 11:34:50.892 Bandwidth: 10485800
15.06.2009 11:34:51.204 RTSP/1.0 200 OK
15.06.2009 11:34:51.204 CSeq: 2
15.06.2009 11:34:51.204 Date: Mon, 15 Jun 2009 07:34:51 GMT
15.06.2009 11:34:51.204 Session: 438785-1
15.06.2009 11:34:51.204 Set-Cookie: cbid=jfhgchjijgifhldmeonogpnqpojrktlufkfjkidlfjdkhiplqnnohpoqqronkqpujfrgghci;path=/;expires=Thu,31-Dec-2037 23:59:59 GMT
15.06.2009 11:34:51.204 vsrc: http://169.229.131.16:8080/viewsource/template.html?nuyhtglloez6x2xitgkp9hz6fe0aE6CAzwqe71w8y62vAty7w63057ujrt79jh7mngahxdE6lDlkh8oE29sep2c5E68f46h7s5hx20gp7ob45mA9sqwgsj5gcutyB6D50to2EcDDmAuhBs6p5wnaCh3hucj8yzr7AyvknEar4ihtCyfr7ydvdkerjnqvjjc2djd0v6ek45l44a27iq8wigun4lCvlvd5h9libhwc9aehh2nxC8AjBs3pwpa93q19AA1zmgyxhyzrB9rcllbs
15.06.2009 11:34:51.204 Last-Modified: Thu, 31 Aug 2006 22:35:28 GMT
15.06.2009 11:34:51.204 Content-base: rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39/
15.06.2009 11:34:51.204 ETag: 438785-1
15.06.2009 11:34:51.204 Session: 438785-1
15.06.2009 11:34:51.204 Vary: User-Agent, ClientID
15.06.2009 11:34:51.204 Content-type: application/sdp
15.06.2009 11:34:51.204 x-real-usestrackid: 1
15.06.2009 11:34:51.204 Content-length: 4525
15.06.2009 11:34:51.235 Total time 01:26:08.549; max bit rate 220000; average bit rate 220000.
15.06.2009 11:34:51.251 SETUP rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39/streamid=0 RTSP/1.0
15.06.2009 11:34:51.251 CSeq: 3
15.06.2009 11:34:51.251 RealChallenge2: c8cdc640f75ca0b604b74f5dc68d0e6501d0a8e3, sd=ccfa04c0
15.06.2009 11:34:51.251 RDTFeatureLevel: 3
15.06.2009 11:34:51.251 Transport: x-pn-tng/tcp;mode=play,RTP/AVP/TCP;unicast;mode=play
15.06.2009 11:34:51.251 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
15.06.2009 11:34:51.251 If-Match: 438785-1
15.06.2009 11:34:51.501 RTSP/1.0 200 OK
15.06.2009 11:34:51.501 CSeq: 3
15.06.2009 11:34:51.501 Date: Mon, 15 Jun 2009 07:34:52 GMT
15.06.2009 11:34:51.501 Session: 438785-1
15.06.2009 11:34:51.501 Reconnect: true
15.06.2009 11:34:51.501 RDTFeatureLevel: 3
15.06.2009 11:34:51.501 StatsMask: 7
15.06.2009 11:34:51.501 RealChallenge3: c5246f2efe3c2dc00cfe77b12ca6b72f4f213d09,sdr=c6f2072b
15.06.2009 11:34:51.501 Transport: x-pn-tng/tcp;interleaved=0
15.06.2009 11:34:51.532 SETUP rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39/streamid=1 RTSP/1.0
15.06.2009 11:34:51.532 CSeq: 4
15.06.2009 11:34:51.532 RDTFeatureLevel: 3
15.06.2009 11:34:51.532 Transport: x-pn-tng/tcp;mode=play,RTP/AVP/TCP;unicast;mode=play
15.06.2009 11:34:51.532 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
15.06.2009 11:34:51.532 Session: 438785-1
15.06.2009 11:34:51.767 RTSP/1.0 200 OK
15.06.2009 11:34:51.782 CSeq: 4
15.06.2009 11:34:51.782 Date: Mon, 15 Jun 2009 07:34:52 GMT
15.06.2009 11:34:51.782 Session: 438785-1
15.06.2009 11:34:51.782 RDTFeatureLevel: 3
15.06.2009 11:34:51.782 StatsMask: 7
15.06.2009 11:34:51.782 Transport: x-pn-tng/tcp;interleaved=1
15.06.2009 11:34:51.798 SET_PARAMETER rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39 RTSP/1.0
15.06.2009 11:34:51.798 CSeq: 5
15.06.2009 11:34:51.798 Subscribe: stream=0;rule=2,stream=0;rule=3,stream=1;rule=12,stream=1;rule=13
15.06.2009 11:34:51.798 Session: 438785-1
15.06.2009 11:34:51.798 PLAY rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39 RTSP/1.0
15.06.2009 11:34:51.798 CSeq: 6
15.06.2009 11:34:51.798 User-Agent: RealMedia Player HelixDNAClient/10.0.1.338 (win32)
15.06.2009 11:34:51.798 Session: 438785-1
15.06.2009 11:34:51.798 Range: npt=433.010-5168.549
15.06.2009 11:34:52.048 RTSP/1.0 200 OK
15.06.2009 11:34:52.048 CSeq: 5
15.06.2009 11:34:52.048 Date: Mon, 15 Jun 2009 07:34:52 GMT
15.06.2009 11:34:52.048 Session: 438785-1
15.06.2009 11:34:52.064 Start receiving data.
15.06.2009 11:34:52.110 RTSP/1.0 200 OK
15.06.2009 11:34:52.110 CSeq: 6
15.06.2009 11:34:52.110 Date: Mon, 15 Jun 2009 07:34:52 GMT
15.06.2009 11:34:52.110 Session: 438785-1
15.06.2009 11:34:52.110 RTP-Info: url=rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39/streamid=0;seq=0;rtptime=0, url=rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39/streamid=1;seq=0;rtptime=0
15.06.2009 11:34:52.110 Range: npt=429.121000-5168.548000
15.06.2009 11:34:52.110 RTCP-Interval: 250
15.06.2009 11:34:52.110 x-predecbufsize: 306000

Originally Posted by any ANONYMOUS forum user

The course link is

http://webcast.berkeley.edu/course_details.php?seriesid=1906978357

I right clicked on the page and selected "Download all files using NetXfer" and then chose only files of type=real but the initial files (.rm) get downloaded which are 88kb whereas the rtsp links are not getting downloaded.

You may use the following HTML file to download all lectures from this course using "Download All":

<a 
href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061211.rm?start=00:09:45&end=01:31:19">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061207.rm?start=00:01:48&end=01:00:17">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060908.rm?start=00:10:27&end=01:31:18">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060911.rm?start=00:2:59&end=01:25:35">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061116.rm?start=00:05:20&end=01:18:54">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061121.rm?start=00:03:24&end=01:16:24">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061128.rm?start=00:05:58&end=01:22:32">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061130.rm?start=00:03:29&end=01:24:16">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061114.rm?start=00:02:03&end=01:21:17">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061031.rm?start=00:06:10&end=01:22:08">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061102.rm?start=00:03:06&end=01:23:04">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061107.rm?start=00:07:11&end=01:17:10">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061019.rm?start=00:00:53&end=01:19:40">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061024.rm?start=00:04:22&end=01:18:28">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061026.rm?start=00:05:45&end=01:21:49">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061010.rm?start=00:02:26&end=01:22:09">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061012.rm?start=00:00:51&end=01:19:31">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061017.rm?start=00:03:45&end=01:20:00">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060921.rm?start=00:01:25&end=01:16:20">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060926.rm?start=00:03:42&end=01:23:31">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061003.rm?start=00:04:04&end=01:18:44">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20061005.rm?start=00:00:01&end=01:17:56">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060919.rm?start=00:00:54&end=01:21:59">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060907.rm?start=00:02:26&end=01:22:05">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060914.rm?start=00:02:09&end=01:02:27">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060905.rm?start=00:01:55&end=01:23:00">lecture</a><a href="rtsp://169.229.131.16:554//classes/f2006/ee105/20060831.rm?start=00:07:13&end=01:23:39">lecture</a>

[thisisandu]

I shall try it out and let you know

thanks for the help

-------update----------

The html doesnt seem to work for me

I do hope Net transport will work for berkeley webcast