Export Question

[sdunnin]

I was wondering why the export adds an "http://" and "https://" version for every TLD entry in my whitelist when they don't seem to exist in the actual whitelist.

[Tom T.]

Unable to reproduce, using both the Whitelist-only export and the "Export everything" (lower) button.
The TLDs with no protocol (which are at the top of the w/l) still have no protocol in the exported file, and those with a protocol (found in the bottom of the NS Whitelist) retain the protocols.

Not sure if it would make a difference, but just out of curiosity, are you exporting these as *.txt files, or in some other format? (I use .txt)

Can anyone else reproduce OP's issue?

[GµårÐïåñ]

Its an internal redundancy to ensure that there is not inconsistencies between the regular http and ssl secured ones to ensure that in specifically configured settings the right rule is applied. When you allow something in the list, it only shows you what you have allowed as an entity unless HTTPS always or something like that is enforced in which case it needs to check to see if it also exists on the HTTPS rule and is valid or not so it can enforce accordingly. Think of it as a shadow list to be applied to a very specific and restrictive configuration.

@Tom, not really an issue per se as I explained to the OP. It used to be that you could see it in about:config but it has not been showing up there for a while now due to the structure of the storage changes. But on export you can see it, for example here is a sample export I generated from a dev profile:

Code: Select all

akamaihd.net
amazonaws.com
apis.google.com
bing.com
cisco.com
cloudfront.net
digitalchocolate.com
facebook.com
facebook.net
farmville.com
fbcdn.net
googleapis.com
imdb.com
live.com
media-imdb.com
mozilla.net
mozilla.org
netacad-cdn.net
netacad.net
nzhost.me
plusone.google.com
scribd.com
scribdassets.com
socialreviver.net
spocklet.com
tetrisfb.com
tp-cdn.com
twitter.com
zynga.com
about:blank
about:newtab
http://akamaihd.net
http://amazonaws.com
http://bing.com
http://cisco.com
http://cloudfront.net
http://digitalchocolate.com
http://facebook.com
http://facebook.net
http://farmville.com
http://fbcdn.net
http://googleapis.com
http://imdb.com
http://live.com
http://media-imdb.com
http://mozilla.net
http://mozilla.org
http://netacad-cdn.net
http://netacad.net
http://nzhost.me
http://scribd.com
http://scribdassets.com
http://socialreviver.net
http://spocklet.com
http://tetrisfb.com
http://tp-cdn.com
http://twitter.com
http://zynga.com
https://akamaihd.net
https://amazonaws.com
https://bing.com
https://cisco.com
https://cloudfront.net
https://digitalchocolate.com
https://facebook.com
https://facebook.net
https://farmville.com
https://fbcdn.net
https://googleapis.com
https://imdb.com
https://live.com
https://media-imdb.com
https://mozilla.net
https://mozilla.org
https://netacad-cdn.net
https://netacad.net
https://nzhost.me
https://scribd.com
https://scribdassets.com
https://socialreviver.net
https://spocklet.com
https://tetrisfb.com
https://tp-cdn.com
https://twitter.com
https://zynga.com
[UNTRUSTED]
bkrtx.com
cotssl.net
google-analytics.com
googlesyndication.com
googletagmanager.com
googletagservices.com
infolinks.com
pubmatic.com
quantserve.com
rlcdn.com
rubiconproject.com
socialvi.be
trialpay.com
http://bkrtx.com
http://cotssl.net
http://google-analytics.com
http://googlesyndication.com
http://googletagmanager.com
http://googletagservices.com
http://infolinks.com
http://pubmatic.com
http://quantserve.com
http://rlcdn.com
http://rubiconproject.com
http://socialvi.be
http://trialpay.com
https://bkrtx.com
https://cotssl.net
https://google-analytics.com
https://googlesyndication.com
https://googletagmanager.com
https://googletagservices.com
https://infolinks.com
https://pubmatic.com
https://quantserve.com
https://rlcdn.com
https://rubiconproject.com
https://socialvi.be
https://trialpay.com

(Notice the domain only, http, and https iterations)

[sdunnin]

Might I suggest that functionality of the export be made optional if in fact it is meant to cover very specific scenarios. If there are many entries in one's whitelist these extra entries that are added can make the list very large over time. I use the export / import to keep my whitelist in sync between multiple computers and all of the extra entries are starting to make for much larger files.

[GµårÐïåñ]

Might I suggest that functionality of the export be made optional if in fact it is meant to cover very specific scenarios. If there are many entries in one's whitelist these extra entries that are added can make the list very large over time. I use the export / import to keep my whitelist in sync between multiple computers and all of the extra entries are starting to make for much larger files.

It does not affect the performance of the browser or the addon, hence why it is done this way. Eliminating would actually put a larger overhead burden on the system and be more negatively impacted. For the purposes of export/import, the program captures the actual and true settings of the extension and therefore all settings are taken EXACTLY as they are in production, so that means having the appropriate and functionally necessary items present. Even if you were to remove them, they would probably get recreated which can impact import time and at worst could expose you to holes in your security. The best practice has been opted for here and you should just make your peace with it.

[sdunnin]

As a software developer myself I'm not sure that I would completely agree with the performance implications of a smaller or larger import file, however, I'm in no position to argue the fact as I have no knowledge of how the code works. It also does not seem like eliminating that from the export/import would negatively impact the performance of the browser/addon since it sounds like they are always added internally anyway.

Just an observation and opinion, no flaming intended.

Thanks for your feedback.

[GµårÐïåñ]

That's why I said "MAY" cause holes in security. Although every precaution is taken to ensure that rules are applied properly in every instance, its the nature of the beast, as a developer you should realize that, that when incomplete or improper settings are imported into a program, despite the efforts of the developer to catch all instances of misconfiguration or issues, there is always a chance that providing incomplete settings would cause an application to misfire when encountering a condition that was not properly placed in its settings. As to those implications, I will allow Giorgio to comment himself but in private, he has been in agreement with me on this. I was hoping that if the perspective was wrong or didn't apply he would comment, he told its being handled fine.

Now as to the performance issue. We have said repeatedly, Giorgio has commented on this as well, unless you are running in the thousands of rules/surrogates/abe/etc, you will not see any significant impact because the iteration loops, which you should be familiar with, fire at the rate of nanoseconds, not even milliseconds, unless you are running a real old dinosaur of a machine with P2 on it in which case you might see about a 10 seconds (for a thousand items) delay, even then, its pretty negligible. Plus, if you are having to deal with a list in the thousands, hell even more than a couple hundred at most, there is something behaviorally wrong with how you browse the web and use the tool. To each their own but even the most paranoid and anal retentive security conscious people I have met in my years on this project have barely scratched the couple hundred mark and you won't see even a hiccup until you reach in the thousands. Even then, you are looking at a momentary pause per k on machines with at least P4 or higher which less lag than the browser itself has been creating recently due to crappy coding by the Mozilla team.

One last thing, unless you are unreasonable, rude, or trolling, you will never get flamed or attacked on this forum, we are here to help and make people understand their situation if necessary, we can't do that without questions and dialog, so no worries there. You have every right to your opinion and we will do our best to answer them in such a manner that helps you get closure on them. IN the end, if you still don't agree, that's fine, we'll take a look and see if there is something we are missing and if not, we move on and so will you and things will be just fine.

[Tom T.]

Sorry to get between you two, but I still show that the export list does *not* add protocols to the raw form of the export list.

When I imported the same list back into Whitelist, the w/l appeared the same as before: no-protocol listings at the top; those with specifically http or https at the bottom. Nothing changed. So I still don't understand what is the problem here or how to reproduce it.

For what it's worth, this reflects *exactly* how these are stored in prefs.js file: no-protocol listing first, then a new alphabetized list of protocol-specific.

OP: Care to post your exported whitelist, copy/pasting into code tags here? Feel free to remove any that you feel might compromise your privacy.

[GµårÐïåñ]

@Tom, it DOES and I posted a clipping of it, take a look by scrolling down/up depending on your configuration (http://forums.informaction.com/viewtopi ... 678#p47469)

[Tom T.]

@Tom, it DOES and I posted a clipping of it, take a look by scrolling down/up depending on your configuration (http://forums.informaction.com/viewtopi ... 678#p47469)

Your list has apis.google.com on the top, but not on the bottom with a protocol. Under the Black Swan Fallacy, your theory is disproven.

To the extent that it appears to occur, this would seem to be a recent change in NS export behavior. However, I just did a test case, whitelisting a site I'd never w/l before, exporting, then inporting the export. No http or https showed up in the imported list.

Perhaps Giorgio needs to look at this.

[GµårÐïåñ]

I have already asked him to take a look and comment. He told me that my position was correct. Perhaps he will take another look. It is not a fallacy as you stated, as the http and https protocols do appear but they seem to focus on those with a WWW precursor. api.google.com is not such a domain. But who knows, maybe it has been changed.