Page 1 of 1
course materials from gated sites
Posted: Sat Apr 18, 2009 9:01 pm
by SandorL
Hi all,
my main use for FlashGot would be to get all course material from my school's website. However, the site is gated, and this must be the reason that even if I want to run FlashGot when I am already logged in (and even the 'remember me' option is checked in, though one session should work anyway), all that FlashGot gets is authenticate.html links instead of the real pdf documents linked in (which a browser opens, of course). Is there a way around this?
Thanks,
Laszlo
Re: course materials from gated sites
Posted: Sun Apr 19, 2009 4:18 pm
by Giorgio Maone
What download manager are you using FlashGot with?
Re: course materials from gated sites
Posted: Mon Apr 20, 2009 5:28 am
by SandorL
I tried simply the browser's and DownThemAll too. If it's their fault, I'm sorry. I just got suspicious as the browser itself would eventually open all the pdf files (most likely after some redirects through these authenticate.html pages), and thought that downloading should go similarly.
Thanks for paying attention.
Re: course materials from gated sites
Posted: Mon Apr 20, 2009 2:41 pm
by therube
If
dTa does not support http
s (ssl) then that may be where your problem lay?
Give a different downloader a shot & see if that works for you.
Re: course materials from gated sites
Posted: Tue Apr 21, 2009 1:03 am
by SandorL
No, sorry, that's not the problem. At least it did not work with Leech either. Is it simply a feature that download managers are missing? What authentication/redirect does my browser do if I click on a link after logging in that download managers cannot? Links are like this:
http://isites.harvard.edu/fs/docs/icb.t ... Ch11_1.pdf
Maybe this helps: the site is full of icb.do links, whatever they do. Sorry that I don't know.
I appreciate your help a lot, thanks.
Re: course materials from gated sites
Posted: Tue Apr 21, 2009 2:19 am
by GµårÐïåñ
Have you tried using something like
Offline Explorer, I have used it successfully with SSL, HTTPS authentication, gated streaming videos from law school archives with no problems whatsoever and it downloads them concurrently and FAST. Not free though, that's the only drawback.
Re: course materials from gated sites
Posted: Tue Apr 21, 2009 8:43 pm
by SandorL
Thanks. But this won't run on Mac OS X.
Do you really think that it is a feature that only the select few support among download managers?
And why wouldn't the browser's own downloader download it (assuming FlashGot could fetch what it needs) if it could open it?
Re: course materials from gated sites
Posted: Tue Apr 21, 2009 8:54 pm
by Giorgio Maone
@SandorL:
could I see the download workflow? Would you mind to send me (temporary) credentials to check by PM or email?
Maybe they use a META refresh or a JavaScript redirection, which is not supported by any downlod manager (neither internal nor external).
Re: course materials from gated sites
Posted: Tue Apr 21, 2009 10:30 pm
by GµårÐïåñ
SandorL wrote:Thanks. But this won't run on Mac OS X.
Do you really think that it is a feature that only the select few support among download managers?
And why wouldn't the browser's own downloader download it (assuming FlashGot could fetch what it needs) if it could open it?
Sorry, alot of things are not supported by Mac.
Re: course materials from gated sites
Posted: Mon May 11, 2009 6:00 am
by Guest
Unfortunately I have to use a Blackboard website as well. I have a problem, that wget does not use the correct smaller filenames when writing output to the hardrive,
as the regular build-in firefox download manager does.
Filenames should like like this:
"Week1_Macro_Tut.pdf".
Instead, wget saves my files as:
Week1_Macro_Tut.pdf?bsession=56046877&bsession_str=session_id=56046877,user_id_pk1=215521,user_id_sos_id_pk2=1,one_time_token=.
*. BB looks like ASP scripting.. highly appreciate any tips, because its the sole reason for using F'got, same reason as topic starter I guess
Re: course materials from gated sites
Posted: Mon May 11, 2009 8:02 am
by GµårÐïåñ
I have also used Blackboard and they are in my opinion a very bulky, poorly written architecture which is slow and inefficient but just the same if there is a way around this problem therube or Giorgio will help you out. Giorgio, if you need more technical information about blackboard or credentials to log in and test it, let me know; I still have a lingering student account with access.
Re: course materials from gated sites
Posted: Mon May 11, 2009 11:28 am
by Guest
All BB-related problems are fixed if you switch to the the browser built-in download mode.
Awesome piece of software btw. ~~
@ guardian: the only way BB "rocks", is when rocks are attached to its feet in the
deepest part of a very deep lake.
Re: course materials from gated sites
Posted: Mon May 11, 2009 6:34 pm
by GµårÐïåñ
Guest wrote:@ guardian: the only way BB "rocks", is when rocks are attached to its feet in the
deepest part of a very deep lake.
Agreed, it really leaves alot to be desired. Believe it or not WebCT (which they took over and absorbed) was not pretty but was much more efficient and effective than this new system built on .NET, which gives .NET a bad name since I know as a developer it can do SO MUCH better. Oh well.
Re: course materials from gated sites
Posted: Tue May 12, 2009 8:09 pm
by therube
There are a number of "name" related Preference items that may affect the naming that you get with wget.
Might mess around & see if anything helps.
Code: Select all
+ FNAME file name placeholder support in custom
download manager arguments templates
+ file name replacement support for downloads coming
from FlashGot Media on aria2, cURL and wget
(these seemingly only apply to FlashGot Media?)
+ flashgot.dmsopts.dmname.stripHash preference to
remove hash from URLs for download managers messing
with it (enabled by default for Net Transport 2)
+ flashgot.media.guessName preference to control
FlashGot Media file name extrapolation
+ flashgot.media.forceNameHack preference to control
hash-based hack to force extrapolated file name on
external download managers
Re: course materials from gated sites
Posted: Wed Oct 07, 2009 8:06 pm
by Blackboard victim
I had the same problem when using Browser Built-in downloading. Wget downloading has its own problems, as has been pointed out already.
After allowing all cookies, even from third-party websites, I could properly download the files. Although it seems that your Blackboard session could be forcefully terminated upon downloading too many files at a time. You can avoid such problems by, for example, using the file extension whitelist to minimize the amount of parallel connections.