Any way to download PBS videos?

jfniss · Post by **jfniss** » Fri Nov 04, 2011 3:33 pm

Is there any way to download PBS videos, such as http://video.pbs.org/video/2164129717? They seem to be .smi files, and I get an error message saying the file can't be opened. This video was preceded with a 10-second commercial, which did download as an .mp4, but then FlashGot got stuck on the .smi file.

Post by **GµårÐïåñ** » Fri Nov 04, 2011 10:18 pm

I was able to manually hack the link to the video path to this:

Code: Select all

http://www-tc.pbs.org/video/media/swf/PBSPlayer.swf?20428&embed=true&start=0&width=512&height=288&reporting_url=http://log.pbs.org/log/cove-flash-player/&session_id=50fe067d3665128341b162c47c667d44&video=http://video.pbs.org/videoPlayerInfo/2163508920

but unfortunately because it relies on a temporary session id:

Code: Select all

aes$F0D328A3F09FFDA4E6AB6A2DD9AB2228$9jx5iLTPtU2yLhOns+geA88KsIE7yndg7Uc5dApbYTEDcUHlCrDcHy7ItP7PjHsTyg3I5JI3uebBisZ6QqQMfWFCb2dCq9DswR99L7vhtX/BJwq26R7VZnKI6skAJM4ideXWuJQb2yZfs0qKC02pvNByBeVjelMDfKNMwd8bTPFpg2HCySzrTYlFez92LtYXaI9YvH/k4uGGzEcY/O1XLQFDJflFfnHRh+TkV7tbdrSY1oV2DKqyi3Rm9bN7cPfi

by the time I hack through the code to get there and get all the pieces to construct the path, it has expired and so it will show an error saying, sorry we can't load that video right now, due to technical difficulties. If the process was automated and using scripting to do it quickly and almost instantly, we should be able to parse the capture path.

If Giorgio can find the time and wishes to give it a shot, I am sure he can probably find a way to make it work in FlashGot but I got nothing more than the manual hack which is hard to gather before the path expires, not technically, but due to the many steps involves to parse each piece.

Forgot to add, that all that said, if you look at this : http://filext.com/file-extension/SMI you will see that the method they are using for that portion of the video (this additional reading might help too : http://en.wikipedia.org/wiki/Synchroniz ... n_Language) and it might on the back end be using RTMP/RTSP? and that would also cause FlashGot not to work with it. So analyzing that file (if you can give us the path) would allow us to see the way they have set it up to load and can see what protocol is in use.

al_9x · Post by **al_9x** » Sat Nov 05, 2011 2:01 am

jfniss wrote:Is there any way to download PBS videos

Yes, but not with flashgot. PBS videos are served over RTMP, which flashgot can't intercept. You need rtmpdump.

Using rtmpdump can require sniffing the rtmp server host name and redirecting it to localhost, to allow an included local dummy rtmp server or proxy to intercept and show the exact rtmp request from the player, but in this case a somewhat simplified procedure suffices.

You need to view the smil file that flashgot is detecting. Here flashgot could be more helpful, the media icon won't let you copy the smil url, so you need to use httpfox to sniff the contents.

the beginning of the smil file:

Code: Select all

<smil xmlns="http://www.w3.org/2001/SMIL20/Language" xmlns:tp="http://xml.theplatform.com/mps/metadata/content/custom">
	<head>
		<meta base="rtmp://sdwfchwqitj2v.cloudfront.net/cfx/st/" />
	</head>
	<body>
		<ref src="PBS_CP_The_NewsHour/Taxonomy.mp4?player=portal&mediaID=2164129717"

you need to combine the two url components, note the "&" entity needs to be replaced by "&"

Code: Select all

rtmp://sdwfchwqitj2v.cloudfront.net/cfx/st/PBS_CP_The_NewsHour/Taxonomy.mp4?player=portal&mediaID=2164129717

the simplified rtmpdump command that still works:

Code: Select all

rtmpdump -r "rtmp://sdwfchwqitj2v.cloudfront.net/cfx/st/PBS_CP_The_NewsHour/Taxonomy.mp4?player=portal&mediaID=2164129717" -o "local file.flv"

jfniss · Post by **jfniss** » Sun Nov 06, 2011 2:55 pm

Thanks to you both, and especially to al_9x (who seems to have the soluition to my query) for your fast and serious replies. Unfortunately, I'd need "RTMPDUMP for Idiots." I'm not a skilled hacker--just a retired lawyer who volunteers teaching English to adult immigrants and would like to use PBS videos as a teaching tool. For a non-hacker, however, I'm pretty good with my computer. So I tried following al_9x's instructions. I used his link to the rtrmpdump site, downloaded the zip file rtmpdump-2.1-windows, unzipped it, and double-clicked on rtmpdump.exe. Then I tried running al_9x's "simplified rtmpdump command." Here's what I got:

C:\Documents and Settings\James Niss\My Documents\Downloads\rtmpdump-2.1-windows
>rtmpdump -r "rtmp://sdwfchwqitj2v.cloudfront.net/cfx/st/PBS_CP_The_NewsHour/Tax
onomy.mp4?player=portal&mediaID=2164129717" -o "local file.flv"
RTMPDump v2.1
(c) 2009 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...

Then nothing more happened. So I tried running al_9x's longer command, and here's what I got:

C:\Documents and Settings\James Niss\My Documents\Downloads\rtmpdump-2.1-windows
>rtmp://sdwfchwqitj2v.cloudfront.net/cfx/st/PBS_CP_The_NewsHour/Taxonomy.mp4?pla
yer=portal&mediaID=2164129717
'rtmp:' is not recognized as an internal or external command,
operable program or batch file.
'mediaID' is not recognized as an internal or external command,
operable program or batch file.

In the unzipped folder there are three other .exe files: rtmpsrv.exe, rtmpsuck.exe, and streams.exe. Do I need to run those too?

In any case, thanks again for your effort.
Jim Niss
NYC

jfniss · Post by **jfniss** » Sun Nov 06, 2011 6:48 pm

After my last post, I realized that I didn't download the latest version of RMPTDUMP. So I repeated the same procedure with version 2.3, and it worked! Now I have to try extracting the path from another .smil file. I may need more help, but at least, there's progress.

Thanks again.

Post by **therube** » Sun Nov 06, 2011 7:53 pm

Net Transport (shareware, its URL Sniffer component), is able to see & then download the file.

Sound was fine, but playback didn't really happen in MPlayer.
Media Player Classic played both the audio & video just fine.
WMP wouldn't play it at all.

(Assume it has to do with the particular encoding used in the file? WMP can play <other> .mp4 with the right codec installed.)

Code: Select all

rtmp://sdwfchwqitj2v.cloudfront.net:1935/cfx/st/<break>mp4:PBS_CP_The_NewsHour/Taxonomy.mp4?player=portal&mediaID=2164129717	http://video.pbs.org/video/2164129717	http://www-tc.pbs.org/video/media/swf/PBSPlayer.swf?20428

May be able to find a free downloader to capture it too? (Not all downloaders correctly capture everything, & RTMP seems to be particularly troublesome.)

al_9x · Post by **al_9x** » Sun Nov 06, 2011 11:10 pm

jfniss wrote:After my last post, I realized that I didn't download the latest version of RMPTDUMP. So I repeated the same procedure with version 2.3, and it worked!

The latest windows binary is http://rtmpdump.mplayerhq.hu/download/r ... -win32.zip

al_9x · Post by **al_9x** » Mon Nov 07, 2011 8:24 am

possible flashgot improvements

Allow the copying of media urls (in this case the smil)
better smil parsing: currently flashgot extracts all url looking attributes from the smil, missing the media url which is combination (at least in this case) of "<meta base=" and "<ref src="
the extracted rtmp url would be useful for copying, submitting to the download manager (evidently some can download them), and possibly even launching rtmpdump directly. Some servers require the exact reproduction of the rtmp request, but for others just the url will work.

jfniss · Post by **jfniss** » Mon Nov 07, 2011 2:17 pm

A friend of mine tried downloading the PBS video with StreamTransport, and it worked seamlessly--it parsed the .smil file just fine. So I installed it on my usual computer running XP Home, and I couldn't get it to work. When StreamTransport's browser window displayed the PBS Website and tried to run the video, it said, "You need to install Flash Player 11.0." But the Flash was already installed, and there seemed to be no way to link it to StreamTransport. So I tried installing StreamTransport on another computer of mine, this one running XP Pro. It worked, and downloaded the PBS video without a problem. If anyone can explain why StreamTransport can't access the Flash Player on the computer I usually use, I'd be grateful.

Post by **therube** » Mon Nov 07, 2011 8:34 pm

Do you have Flash installed for both IE & mozilla?
(They are separate & different.)

Going to assume that StreamTransport relies on the IE rendering engine & therefor uses the IE version of Flash (Flash11c.ocx).

jfniss · Post by **jfniss** » Wed Nov 09, 2011 3:07 pm

StreamTransport seems to require the ActiveX version of the Flash Player, which is not installed on my everyday computer. I tried but failed to get it installed by checking the firewall and security settings and then reinstalling the Flash Player downloaded from Adobe. Could the problem be that Firefox is my default browser, so ActiveX won't install? In any case, this is no longer a priority for me because I can run StreamTransport on another home computer which has Flash ActiveX installed (but also has Firefox as its default browser--go figure).

Post by **therube** » Thu Nov 10, 2011 9:36 pm

Could the problem be that Firefox is my default browser

Only indirectly in that if you were to go to http://get.adobe.com/flashplayer/ from within FF, Adobe sniffs your browser & by default gives you the Netscape/Mozilla version, which you already have.

Similarly if you were to go there in IE, you would be served the IE version.

But that does not mean that you cannot get the IE version from with FF (or vice-versa).

An easy way (which Adobe does not generally make easy):

Code: Select all

IE (ActiveX, ~3.5 or 7.5 MB respectively):
http://fpdownload.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_11_active_x_32bit.exe
http://fpdownload.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_11_active_x_64bit.exe

Netscape (Mozilla, Opera & all the rest; DLL, ~3.5 or 7.5 MB respectively):
http://fpdownload.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_11_plugin_32bit.exe
http://fpdownload.macromedia.com/get/flashplayer/current/licensing/win/install_flash_player_11_plugin_64bit.exe

Save to your computer & run.

(Heh, I now I see Adobe just updated to version 11.1.102.55.)

InformAction Forums

Any way to download PBS videos?

Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?

P.S. to my last post: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: P.S. to my last post: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?

Re: Any way to download PBS videos?