[RESOLVED]Youtube chunks & encodes videos to avoid downloads

Ask for help about FlashGot, no registration needed to post
g7IITtxxJpRC7J9E
Posts: 1
Joined: Fri Jun 21, 2013 11:19 am

[RESOLVED]Youtube chunks & encodes videos to avoid downloads

Post by g7IITtxxJpRC7J9E » Fri Jun 21, 2013 11:37 am

For videos like this youtube is sending chunks to the player with range parameters encoded in the URL. I found a tutorial at http: code.google.com/p/youtubesniffer/wiki/Tut1UsingNetworkSniffer to download them and it works. If only it was more automated in a button rather than some manual tedious typing... ;)

Oh yeah, antispam filters, nice, even for registered users...

Ooops, something in your posting triggered my antispam filter...
Please use the "Back" button to modify your content and retry.

Are you tired of "saving as..." one by one all those movies, music files, pictures, programs and so on? Are you sick of getting them sooo slowly? Are you afraid of losing them in the middle of download if you get disconnected or close your browser, being forced to repeat all that tedious work?
Well, you need a download manager...

But... Aaaargh!!! my download manager doesn't work with Firefox! They say it does, but where is my Download All menu? How can I handle the tons of mp3, mpeg, jpg files that I want to download?
Well, you definitely need...
Last edited by Thrawn on Thu Aug 29, 2013 10:52 am, edited 2 times in total.
Reason: Fixed truncated link
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/536.30.1 (KHTML, like Gecko) Version/6.0.5 Safari/536.30.1

User avatar
Giorgio Maone
Site Admin
Posts: 8701
Joined: Wed Mar 18, 2009 11:22 pm
Location: Palermo - Italy
Contact:

Re: Youtube chunks and encodes videos to avoid downloads

Post by Giorgio Maone » Fri Jun 21, 2013 12:00 pm

Please notice FlashGot already peforms that kind of "sniffing".
It seems, though, that Youtube slightly changed its behavior for some videos like the ones you linked here, so some code adjustments are needed.
I'll make them in next version, thank you.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0

Mihai Munteanu

Re: Youtube chunks and encodes videos to avoid downloads

Post by Mihai Munteanu » Sun Jun 23, 2013 11:33 am

Doesnt work anymore. YouTube block the Flashgot for a lot of new videos. When you will fix this?
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0

User avatar
therube
Ambassador
Posts: 7426
Joined: Thu Mar 19, 2009 4:17 pm
Location: Maryland USA

Re: Youtube chunks and encodes videos to avoid downloads

Post by therube » Wed Jun 26, 2013 5:27 pm

(just copying, cause probably more appropriate here)

ssjkakaroto wrote:@Giorgio: Do you want some links to videos that flashgot won't detect?


No need for that anymore, thanks, they're quite common now.
Unfortunately it's quite a complex issue (the live stream is not just chunked, but also the audio is split for the video), so I'm still looking for a solution.

from, viewtopic.php?p=55359#p55359
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.19) Gecko/20110420 SeaMonkey/2.0.14 Pinball NoScript FlashGot AdblockPlus
Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 SeaMonkey/2.21a2

Guest

Re: Youtube chunks and encodes videos to avoid downloads

Post by Guest » Wed Jul 03, 2013 9:21 pm

This youtube video is not displaying the Flashgot icon http://www.youtube.com/watch?v=LrUvu1mlWco
Is this the same issue already discussed? (Note that sometimes an ad is shown and may be downloadable, but obviously, I'm not referring to that.)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0

User avatar
Giorgio Maone
Site Admin
Posts: 8701
Joined: Wed Mar 18, 2009 11:22 pm
Location: Palermo - Italy
Contact:

Re: Youtube chunks and encodes videos to avoid downloads

Post by Giorgio Maone » Wed Jul 03, 2013 11:09 pm

Guest wrote:This youtube video is not displaying the Flashgot icon http://www.youtube.com/watch?v=LrUvu1mlWco
Is this the same issue already discussed? (Note that sometimes an ad is shown and may be downloadable, but obviously, I'm not referring to that.)

VeVo is a different beast: Youtube seems to adopt a proprietary DRM scheme for it and it has not been downloadable for a long time now.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0

Guest

Re: Youtube chunks and encodes videos to avoid downloads

Post by Guest » Thu Jul 04, 2013 1:27 am

Giorgio Maone wrote:
Guest wrote:This youtube video is not displaying the Flashgot icon http://www.youtube.com/watch?v=LrUvu1mlWco
Is this the same issue already discussed? (Note that sometimes an ad is shown and may be downloadable, but obviously, I'm not referring to that.)

VeVo is a different beast: Youtube seems to adopt a proprietary DRM scheme for it and it has not been downloadable for a long time now.


There seem to be some commercial packages (e.g. AllMyTube) that claim to be able to capture VEVO streaming video, so I'm guessing it's technically possible. However, you're saying it's complex enough that it won't be supported by Flashgot (I'm guessing). Perhaps these commercial packages have reverse-engineered the Flash Player DRM mechanism?

Of course one possibility with a fast enough PC and web connection is to simply play the video stream into a (perhaps offscreen) full resolution video buffer while capturing and encoding it back to some format (e.g. mp4). With that approach the player plugin software would have no idea what was going on (in theory) and could not prevent it. Of course, that's a completely different approach from what Flashgot is doing so I imagine it's not even on the radar.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0

Calx
Posts: 2
Joined: Sun Jul 14, 2013 9:11 pm

Re: Youtube chunks and encodes videos to avoid downloads

Post by Calx » Mon Jul 15, 2013 10:02 am

I also have this very same problem. It's quite frustrating not able to save your favorite videos.
I like to keep a copy on my external HDDs so I can always watch it whenever I can even after years.
Mozilla/5.0 (masking-agent; rv:18.0) Gecko/20100101 Firefox/18.0

ssjkakaroto
Posts: 19
Joined: Thu Mar 29, 2012 11:51 pm

Re: Youtube chunks and encodes videos to avoid downloads

Post by ssjkakaroto » Tue Jul 16, 2013 12:24 pm

You can use keepvid dot com to download videos that Flashgot won't detect, until Giorgio comes up with a solution.
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:22.0) Gecko/20130620 Firefox/22.0

user

Re: Youtube chunks and encodes videos to avoid downloads

Post by user » Sun Jul 21, 2013 5:28 pm

Just parse ytplayer.config.args.url_encoded_fmt_stream_map or the player's flashvars.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 SeaMonkey/2.19

User avatar
Giorgio Maone
Site Admin
Posts: 8701
Joined: Wed Mar 18, 2009 11:22 pm
Location: Palermo - Italy
Contact:

Re: Youtube chunks and encodes videos to avoid downloads

Post by Giorgio Maone » Tue Jul 23, 2013 7:35 pm

user wrote:Just parse ytplayer.config.args.url_encoded_fmt_stream_map or the player's flashvars.

FlashGot already does it.
It's not simple like that, the "signature" parameter parsed from is encrypted in various ways.
I'm working on it, please be patient because I'm currently traveling and I'm almost always AFK.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0

user

Re: Youtube chunks and encodes videos to avoid downloads

Post by user » Wed Jul 24, 2013 9:14 am

Giorgio Maone wrote:
user wrote:Just parse ytplayer.config.args.url_encoded_fmt_stream_map or the player's flashvars.

FlashGot already does it.
It's not simple like that, the "signature" parameter parsed from is encrypted in various ways.
I'm working on it, please be patient because I'm currently traveling and I'm almost always AFK.


"Various", "not simple", tell me more, please. It's either not encrypted if passed via the "sig" parameter, or this if passed via the "s" parameter:

Code: Select all

var signature = stream.sig || decode_s(stream.s);

function decode_s (s) {
    // return s.substr(4, 19) + s.substr(-1) + s.substring(24, s.length - 2);
    var a = s.split("");
    a[23] = a.pop();
    return a.slice(4, -1).join("");
}


Or if you want the original algorithm (the [spoiler] tag, ever heard of it?):

Code: Select all

var Sig = {
    decode: function (s) {
        var a = s.split("");
        a = a.slice(3);
        a.reverse();
        this._swap(a, 63);
        a = a.slice(2);
        a.reverse();
        a = a.slice(1);
        return a.join("");
    },

    _swap: function (a, idx) {
        var tmp = a[0];
        a[0] = a[idx % a.length];
        a[idx] = tmp;
    }
};


And since we're at what FlashGot does, I have to tell you that it does that in the most retarded way possible: it requests the information that's already there. A typical response (read: unnecessary traffic and server load) is ~25KB for a 360p video and ~35KB for a 1080p video.
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 SeaMonkey/2.19

User avatar
Thrawn
Senior Member
Posts: 3106
Joined: Mon Jan 16, 2012 3:46 am
Location: Australia
Contact:

Re: Youtube chunks and encodes videos to avoid downloads

Post by Thrawn » Wed Jul 24, 2013 11:53 am

@guest: Please keep your posts more respectful, especially when you're talking to the man who single-handedly maintains this, while also looking after a family, and doesn't get paid for it. If you think that FlashGot's behavior is the worst possible, feel free to submit a patch. But as Giorgio said, he doesn't have time at present to work on it.
Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:22.0) Gecko/20100101 Firefox/22.0

Calx
Posts: 2
Joined: Sun Jul 14, 2013 9:11 pm

Re: Youtube chunks and encodes videos to avoid downloads

Post by Calx » Wed Jul 24, 2013 1:49 pm

This script is what Im using as an alternative.
It picks up the new videos too.

http://userscripts.org/scripts/show/25105
Mozilla/5.0 (masking-agent; rv:17.0) Gecko/20100101 Firefox/17.0

GenericGuest

Re: Youtube chunks and encodes videos to avoid downloads

Post by GenericGuest » Sun Jul 28, 2013 3:04 pm

That's an amazing script, Calx. Thanks for the heads up. I tried it to download VeVo videos, and guess what; it WORKS! Awesome.

I hope Giorgio Maone will take a look at the script when he gets back and perhaps ask the script's author if he could put it in FlashGot. As far as general video downloading goes, FG is just unbeateable. If the author could find a way to circumvent this tricky restriction, FG would be just perfect. :D
Mozilla/5.0 (X11; Linux i686; rv:20.0) Gecko/20100101 Firefox/20.0

Post Reply