InformAction Forums

-------------------------------------------------
-------------------------------------------------
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 - Build ID: 20100914125854

XP Pro Svc Pk 3

Flashgot: ver. 1.2.2.1

Download Manager: Free Download Manager Version 3.0 Build 848 (latest version according to app updater).

Downloader: Browser Built-In.

Site: Imagefap.com; attempted full gallery downloads.

ISP Connection: Dial-up.

Except for upgrade from Flashgot 1.2.2 today to version 1.2.2.1, all information above is a "constant" for purposes of this report.
-------------------------------------------------
-------------------------------------------------
PROBLEM: Flashgot 1.2.2.1 is not downloading the "gallery" from the gallery link on the Imagefap.com's user "gallery page" when I right-click the gallery page link to the entire gallery and select FlashGot Link. Flashgot is, instead, stalling and/or freezing up prior to initiating an actual download. The Flashgot progress meter goes almost to the end on the right but never quite gets there; and the download never starts.

Am I doing something incorrectly? Has the sequence in FlashGot changed? All I've ever done at Imagefap before is to right-click the gallery link and select "FlashGot Link"; and the gallery file pics have always downloaded. Should I be doing something else? Last week, when I was still using FG ver. 1.2.2, this is how I did it, and everything downloaded just fine. Today, when ver. 1.2.2 wouldn't work, I came to the site here and installed ver. 1.2.2.1; but I'm still getting the same results as I did in ver. 1.2.2.

As some substantiation that the download is never actually starting, I noticed that the Firefox status bar at the bottom of the screen never changes over from FlashGot [.........] to the built-in browser downloader text that one can normally click to bring up the Fx downloader window in order to check the download status of each file's progress in the gallery download. I do get the folder chooser dialog box, and it allows me to select a destination folder for the download; but the download never actually begins.

My conclusion is that Imagefap.com has changed some code in order to thwart FlashGot downloads of entire galleries. Since Inagefap is not a pay site, I am simply amazed at the lengths to which the site goes in order to prevent image downloads in a group of images as opposed to "Save As" one-by-one downloads.

If you go to the actual gallery itself, Flashgot 1.2.2.1 will in fact download the single picture from the link contained in the thumbnail for the individual picture.

The log for an attempted full gallery download, which log is very short, is as follows:

-------------------------------------------------
*** FlashGot 1.2.2.1 started at Tue, 12 Oct 2010 05:08:07 GMT ***
Sorting again 71 links
Preprocessing done in ms80900
Starting dispatch
Dispatch done in ms16786
Total processing time: ms97686
-------------------------------------------------

...and that's it as far as a log goes.

So, nothing is apparently happening by way of a full gallery download.

Download Link was "Cages" (near bottom of page) at
http://www.imagefap.com/profile/sabins/galleries

Hyperlink embedded in "Cages" is as follows:
http://www.imagefap.com/gallery/1798120

I tried several other pages and several other gallery link downloads as well, all with the same results.

If this is not a bug, could someone please move this post to the correct forum and tell me what I am doing incorrectly? If it is a bug or a current incompatability, could someone simply congirm for me?

Thanks.

Nuncus

> right-click the gallery link and select "FlashGot Link"; and the gallery file pics have always downloaded

Working here. (Never even knew you could do something like that.)
Perhaps something related to computer "horsepower" (lack thereof?) & dial-up connection?
It does take a bit of time to obtain all the links, but the are obtained & sent to my download manager.

> http://www.imagefap.com/gallery/1798120

FlashGot All takes even longer here, but again is working for me.

Rube, did you check the actual downloaded files against what appears in the gallery itself? With some further attempts, here is what I consistently obtained as results:

1. FlashGot using the "built in" Fx download manager: same as described above, i.e., downloads never start.

2. FlashGot using the Free Download Manager download manager: I get LOTS of files that download; unfortunately, they are from all over the landscape among the user's galleries and are not just the one's from the targeted gallery.

With but one, isolated, exception (noted below), these results are consistent.

This appears to be what happened previously with FlashGot and ImageFap. ImageFap changed something in the way it stores files, etc., and FlashGot was unable to tunnel through to the actual ".JPG" files. You'd get downloads all right; but they would all be PHP files. Georgio was able to fix that fairly quickly; and the subsequent update of FlashGot worked perfectly. It was only last week that the new "break" occurred at ImageFap.

Don't forget: in either scenario, if I go to the actual gallery page and apply Flash Got Link to an individual thumbnail, I DO in fact get the targeted file; and, if I use FlashGot with the "Built-in" Fx download manager, the file downloads with the user's original filename for the file in the gallery instead of ImageFap's renamed filename.

So, it's probably a bit of an overstatement to say that ImageFap has broken FlashGot entirely because it hasn't exactly done that.

What's not working is when you apply Flash Got Link to the entire gallery link.

Note also: I did in fact get a download on ONE attempt using the Fx "built-in;" BUT I got files from all over the landscape from among the ImageFap user's galleries. That's why I am asking whether you checked your downloaded files against what's in the gallery itself. I'd be curious to know if what you got in the download exactly matched what you get if you use the manual and laborious "Save As" method and download the files one at a time.

As I said, if I switch to using FlashGot with the Free Download Manager instead of using the Firefox "built-in" file manager, I get a TON of files; in fact they just keep downloading and downloading and downloading; BUT, they are NOT confined to the specific, target, User directory for the named gallery. Instead, FlashGot seems to be pulling them from multiple directories belonging to the ImageFap user.

Horsepower? I don't think that has anything to do with it. When I do get downloads (using the Free Download Manager with FlashGot) not only do I get "downloads" (although many are not in the target directory), but also the files download fairly rapidly -- not like broadband, but pretty quickly nevertheless.

In fact. when FlashGot is working on ImageFap, and when I use Flash Got Link on a target gallery, by the time I have selected the download folder and then drilled down to it in Windows Explorer, ALL the filenames already have been collected and appear in the Windows folder window.

Now, it does in fact take a while for the actual downloads to complete on dial-up, to be sure, but the downloads generally all complete, except the last four files being downloaded always seem to require a "Retry" to get them down. I've never reported that little annoyance here because it is a very minor inconvenience and really not worth the time to complain about.

No, it ain't "horsepower;" for some reason, ImageFap seems to have gotten its arse up in the air over FlashGot; personally, my suspicion is that ImageFap, albeit a "free" site, is deliberately trying to mess up FlashGot's retrieval of entire galleries at once with Flash Got Link.

Best regards,

Nuncus

> Rube, did you check the actual downloaded files against what appears in the gallery itself?

Actually I did not.
I stopped looking once I verified that (what appeared to be valid) links were being generated & sent to my download manager.

> I get LOTS of files that download; unfortunately, they are from all over the landscape among
> the user's galleries and are not just the one's from the targeted gallery

Confirmed, kind of -ish.

Using the above URL, FlashGot All generated 278 (jpg/jpeg) links.
Obviously more then in that two page gallery.

I do get all of the expected files. They are named per the name given on a mouseover.
Like http://www.imagefap.com/image.php?id=1240106307&pgid=&gid=1798120&page=0 is named "1240106307"

I also get thumbnails of all the same files. Sometimes multiple sized thumbs of the same file.

Looks to be 39 pictures in that gallery.
So if the pic + thumb, you're looking at 78 total + a few more for multi-sized thumbs.

And then on top of that, I've picked up roughly another 200, totally unrelated pictures.


If you look at the page source, you will see links to "related galleries". These being totally unrelated to the gallery at hand. Don't know why these thumbs/references don't display? Unless they're only there for "spiders" & "hit counts"? Or unless something on my end is blocking them?

Anyhow it must be those links that FlashGot is also finding & "spidering" it's way down & subsequently downloading to you.

Guess FlashGot's search could be filtered a bit better? Perhaps ignoring links that contain /mini/ (like "<img src="http://x.fap.to/images/mini/39/116/1163749983.jpg"). And only pick up links that have an "<a name" & a relative reference (like "<a name="1853099035" href="/image.php?id=1853099035&pgid=&gid=1798120&page=0">)?

Rube, try the download again and DON'T use "Flash Got All." You shouldn't have to use "Flash Got All."

Just use "Flash Got Link."

The way Georgio had it set up after the last time he fixed the similar previous break by ImageFap was that you simply applied "Flash Got Link" to the link in the ImageFap user's section that lists the user's galleries, and FlashGot would somehow bore in and go get the end point JPG's that connect to the thumbnails on the actual gallery page itself.

"Flash Got All" -- I think -- picks up "everything" on the page, so it makes sense that you'd get hundreds of downloaded files.

Sorry for being vague and ambiguous in my post and using the word, "LOTS;" "lots" is a relative term. I didn't get "hundreds;" I got maybe two or three times the number of files listed in the actual gallery preview page. At some point, realizing that something wasn't going right, I stopped the download; so, I don't really know how many files I would have downloaded if I had just let the downloading run on until it finally stopped of its own volition.

With "Flash Got Link," you shouldn't be getting all those extraneous files. Flash Got Link might pick up five or six ".PHP" files in addition to the end JPG's; but that's about all.

In other words, as per the last fix, if you are using "Flash Got Link," you should be getting the actual JPG end point files plus maybe five or six extraneous ".PHP" files and none of the ".gif" files that are part of the various web pages that might be involved. Don't forget that "Flash Got All" picks up every single image that appears on the "web page."

I'm afraid I can't be of much help on the programming technicals since I've only "used" FlashGot; it's one of those things where the software -- until ImageFap's current hijinx began -- always "just worked" for me. I don't have a clue as to how Georgio's doing this stuff programmatically. It's always been "Maone Magic" as far as I'm concerned.

About all I have to say on the matter is that those Dutch boys and girls are peeing up the proverbial rope if they think they can stump the Pizzon.

I just wanted him to know they're "messin' with him" again.

I'm sure that when he becomes aware of the problem, and as soon as he can find the time, he'll most likely not only get the problem fixed but also have those folk at ImageFap talking backwards to themselves trying to figure out what he's done.

But, in the meantime, I would be curious to know what your results are -- time permitting -- if you repeat your download using "Flash Got Link" with the "built-in" download manager in Fx. When everything works like it is supposed to work, that's the best download combination to use: your bottom line is that you should get all the "end point files," with just a wee bit of trash thrown in, and all the filenames are the ImageFap user's original filenames that he used when he uploaded the gallery files.

It'd be nice to have a confirmation -- yea or nay -- in place for the programmer when he sits down to remedy the situation; in other words, it will most likely save him some preliminary time if he knows he's got at least two users doing the same thing and both are consistently obtaining the same results. ...Of course, if they are not getting the exact same results, well, that would be useful to know, too.

Thanks for taking a look at it. sorry if I was a little bit slow in posting back at you; but I needed to test in a number of galleries to be sure my results were consistent before I started sounding too sure of myself here. My expectation, though, is that, based on what you've reported above, you'll probably get the same -- or close to the same -- results I did if you use "Flash Got Link" with the "built-in."

Best regards,

Nuncus

Note "full". I did get the full sized image.
Note "thumb". All I got were the thumbnails.

(above was with my download manager, Net Transport)


Same results with Browser Built In.
Only difference is with FlashGot Link, the name; download manger = 1240106307.jpg, built in = 36.jpg.

(With FlashGot selection I was careful to only highlight the three thumbnails)


FlashGot Selection is not "drilling down", all it is picking up are the thumbnails.


When I expanded the FlashGot Selection area (to include all the images on the page & some extraneous links too, & using Built In) I go 74 or so images - all thumbnails, some expected, some unexpected. No fullsized images at all.


(Downloading larger numbers with Browser Built In seem inefficient.)

Thanks, Rube. That's useful information. You certainly covered the waterfront.

Just note that you "should" be able to go to "User Profile>User's Galleries" and apply "Flash Got Link" to any link that you would otherwise click to go to the gallery page (the page with the full set of larger thumbnails that link to the end point files themselves), and Flash Got Link "should" download all the end point files themselves, bypassing your ever having to go to the user's actual gallery page itself at all.

Somehow, Georgio had the thing set to where it would go directly from the link to the gallery page into the gallery and pick out the end point files themselves.

If you do that, and assuming FlashGot is working like it's supposed to work on ImageFap, then, ALL the endpoint files, with the ORIGINAL filenames, and not ImageFap's re-names, "should" appear right there in your Built-In Fx/Se Monkey browser's download manager.

That long file name your download coughed up is ImageFap's re-name; the shorter one is the User's original file name. So, FlashGot's not entirely broken on ImageFap; ImageFap just broke some of it this time.

Once the downlad begins, then it's just a matter of connection speed as to how fast the actual files download to your box.

Just also note, you are not going to be able to pick and choose individual files by applying Flash Got Link to the main link to the user's gallery page; it's all or nothing; if you apply Flash Got Link to the user's link to the gallery page, you get ALL the end point files for the entire gallery itself. If you are using FlashGot with Fx's built-in dwnld mngr, about the only way you can pick and choose is by going to the download manager and "removing" a download from the displayed download list of files. If you are using hi-speed, I suspect that, by the time you finish bringing up the dwnld mngr and looking at the list of files, they are probably already in the destination folder on your HDD.

Even on dial-up, the preliminary aggregating of the original file names for the entire gallery in the built-in download manager only takes a few seconds; and the file names themselves appear right away in the destination folder window on your own box.

Of course, the actual download time of the files in the target gallery is going to vary drastically between dial-up and broadband or DSL; but the initial steps, which mainly involve simply retrieving characters, complete a lot faster than I expected them to complete on a dial-up.

Well, I hope our man can get FlashGot to where it bores through the gallery page and then in to the actual end point files again and doesn't get distracted by all the peripheral files. "User galleries>Gallery Page>thumbnail> Save As" for each individual file is not an attractive option.

Again, thanks again for taking the time to go back and give FlashGot another run at the gallery downloads.

One thing you could do that Georgio might find helpful is to go into your FlashGot Options and turn ON FlashGot's logging, delete the log entries, exit the log viewer page, then immediately thereafter begin the download; and then just post either the whole log, or a portion of it. That way, the programmer can actually track FlashGot's progress as it attempts to find the actual end point file.

Since my downloads were never actually starting, unfortunately, my own log I posted above doesn't really tell anybody much of anything except that "nothing happened."

Nuncus

The main issue I could see is that ImageFap now includes an hidden "Related Galleries" section which includes linsk to every single image uploaded by the same user, no matter which gallery it belongs.

This caused FlashGot to "overload" with hundreds of links even if the wanted gallery contained just a couple.

I took the chance to speed-up the pre-processing routine, too.

Please check whether there are other problems in latest development build.

Well, I just now tested it on two galleries. Works like a charm! And downloaded files match what's in the target gallery page. The filenames are all the user filenames and not IFap re-names.

Also, the "hot rod" speedup is a nice touch; by the time I could minimize my Fx and look at the Windows folder window, the filenames were already there.

This plug-in lives in the realm of "MAGIC."

Thanks a bunch, Georigio; I was hardly expecting such a fast fix. In fact, I only checked back in to see if the Rube had posted up a download log yet. I guess that's besides the point, now....

Nuncus

Hi, I didn't think it was worth starting a new thread since I seem to be having the same problem, I have the newest dev version of flashgot and the newest firefox as well as free download manager.

Basically I can batch download some galleries from imagefap and succesfully downloaded the cages one (http://www.imagefap.com/pictures/1798120/cages) nuncus was enquiring about using Flashgot link

but a lot of other galleries (for example http://www.imagefap.com/pictures/255739 ... in-the-ass) just stall flashgot after asking me to select a download location and no files are generated.

Any help would be greatly appreciated.

Please check latest development build, thanks.

Thanks for the speedy response, unfortunately it still isn't working perfectly, A lot of galleries that didn't work seem to work now (some needing a few attempts) but there's a few that i've tried 20+ times since the upgrade and they still aren't working, there doesn't appear to be any common factors that distinguish the ones that will work from those that don't.

Thanks for your help.

ofented wrote:there doesn't appear to be any common factors that distinguish the ones that will work from those that don't.

Having some samples of those not working would help, though.

Ok http://www.imagefap.com/pictures/255739 ... in-the-ass still hasn't worked for me but most of the others I've been trying have now, this one http://www.imagefap.com/pictures/242878 ... e-aug-2010 still hasn't worked. thanks again

ofented wrote:Ok http://www.imagefap.com/pictures/255739 ... in-the-ass still hasn't worked for me but most of the others I've been trying have now, this one http://www.imagefap.com/pictures/242878 ... e-aug-2010 still hasn't worked. thanks again

Both these gallery do work fine for me, but they're very large (over 150 pictures each).
Most likely your network connection is too slow to spider them and FlashGot appears to work for ever (or it just time outs).