[RESOLVED] Functionality of allowWhitelistUpdates changed?

[GuestPassingBy]

[ split from viewtopic.php?f=7&t=20982 - barbaz ]

noscriptService.js, onVersionChanged

Version 2.6.9.26 and some earlier:

Code: Select all

if (!this.getPref("allowWhitelistUpdates")) {
      return;
    }

// update hacks

Version 2.6.9.27 through 2.6.9.30rc3

Code: Select all

let removalsOnly = !this.getPref("allowWhitelistUpdates");

// update hacks

It appears that the change was meant to treat allowWhitelistUpdates as though it were allowWhitelistAdditions. Does that even work? Where is removalsOnly tested?

[barbaz]

It appears that the change

... somewhere along the line has in fact completely disabled the functionality of the pref allowWhitelistUpdates
(I have not tested this to confirm, but it sure looks like it looking at the code...)

Full function, 2.6.9.30rc3:

Code: Select all

  onVersionChanged: function(prev) {
    let removalsOnly = !this.getPref("allowWhitelistUpdates");

    // update hacks
    var versions = {
      "2.1.1.2rc6": {
        "hotmail.com": "wlxrs.com", // required by Hotmail/Live webmail
        "google.com": "googleapis.com gstatic.com", // required by most Google services and also by external resources
        "addons.mozilla.org": "paypal.com paypalobjects.com" // required for the "Contribute" AMO feature not to break badly with no warning
      },

      "2.2.9rc2": {
        "addons.mozilla.org": "persona.org"
      },

      "2.4.9rc2": {
        "!browserid.org": "persona.org"
      },
      "2.5.9rc3": {
        "live.com": "gfx.ms afx.ms" // fully Microsoft-controlled (no user content), now required by MS mail services
      },
      "2.6.5.9rc2": {
        "live.com": "sfx.ms" // fully Microsoft-controlled (no user content), now required by MS mail services
      },
      "2.6.6rc5": {
        "live.com": "outlook.com live.net" // fully Microsoft-controlled (no user content), now required by MS mail services
      },
      "2.6.9.4rc1": {
        "vimeo.com": "vimeocdn.com" // no movie will play anymore without this
      },
      "2.6.9.19rc1": {
        "youtube.com": "googlevideo.com" // Youtube's HTML5 video player now requires this
      },
      "2.6.9.22rc1": {
        "prototypejs.org": "bootstrapcdn.com" // Used by many sites, mostly for styles and fonts
      },
      "2.6.9.27": {
        "!vjs.zendcdn.net": "" // removal
      },
      "2.6.9.28rc2": {
        "!googleapis.com": "ajax.googleapis.com" // storage.googleapis.com allows HTML files!
      },
      "2.6.9.30rc1": {
        "!mootools.net": ""
      },
      "2.6.9.30rc2": {
        "!cdnjs.cloudflare.com": "",
        "!prototypejs.org": "",
        "ajax.googleapis.com": "maps.googleapis.com"
      },
    };

    for (let v in versions) {
      if (this.versionComparator.compare(prev, v) < 0) {
        let cascading = versions[v];
        for (let site in cascading) {
          let newSite = cascading[site].split(/\s+/);
          let replace = site[0] === "!";
          if (replace) site = site.substring(1);
          if (this.isJSEnabled(site)) {
            if (newSite[0]) {
              this.jsPolicySites.remove(newSite, true, false);
              this.setJSEnabled(newSite, true);
            }
            if (replace) this.jsPolicySites.remove(site, true, false);
          }
        }
      }
    }
  },

I'm splitting this to a new thread, thank you for pointing this out.

EDIT Since this needs fixing anyway, I'd like to note viewtopic.php?f=10&t=20965

[GuestPassingBy]

Thanks barbaz.

I performed a few tests: install FF 38.0.5 portable app, install NoScript 2.6.9.29, then install NoScript 2.6.9.30rc3. After the final step I did not detect related changes, and eventually came to think that onVersionChanged wasn't even being called. Correct or not, I'm sure Mr. Maone and/or others will quickly determine what needs attention.

I'd like to suggest that there be controls, in the NoScript Options GUI, which determine whether or not the user receives developer-pushed, site-specific rules and updates to those. I think I would choose to have noscript.mandatory updates made to my whitelist and no other changes made to my whitelist. So I'd suggest that this be one of the supported patterns, and it be something that we can count on (no exceptions).

[barbaz]

I'd like to suggest that there be controls, in the NoScript Options GUI, which determine whether or not the user receives developer-pushed, site-specific rules and updates to those.

Hmm... it's kind of an "advanced" setting, but I guess it could be OK if there's room somewhere in the Advanced section...

I think I would choose to have noscript.mandatory updates made to my whitelist and no other changes made to my whitelist.

There is no option to disable changes to noscript.mandatory unless you set it in user.js yourself or otherwise give it a user-set value, and IMO it should stay that way. The mandatory items are required by the browser itself and there could be major stability problems if it's not changed as it needs to be (for the most part, those entries really aren't optional).

[GuestPassingBy]

I've looked at the noscript.mandatory list and user-set the pref during some testing. Although it might not have been clear, I do think some browser internal pages would deserve to be specified in a pref that is not surfaced in the GUI.

I think there would be a limit to that though. I believe browser internal pages, including those necessary for an advertised feature to work correctly, can communicate with remote servers. Which opens the door to a variety of potential security threats. Most might fall into the information security category, but maybe there are or will be cases where the response has the potential to adversely affect browser behavior.

I noticed v 2.6.9.30rc4 is out, and glanced at noscriptService.js to see if there are an changes related to allowWhitelistUpdates. I don't spot a change. However, I did see this:

"2.6.9.30rc4": {
"about:blank": "about:pocket-signup about:packet-save"
}

Should about:packet-save be about:pocket-saved?

[barbaz]

Not only that but I am against that change for users' existing whitelist. SeaMonkey users do not need those (they point to nothing... what if a malicious extension targeting SeaMonkey adds those URLs)? Furthermore, they're added for those who kept about:blank... but about:blank doesn't depend on these for functionality.

And furthermore, they were added to my whitelist when I opted out of whitelist changes!!!!!
I'm kind of starting to lose trust in NoScript's default whitelist, and I don't like it. I'd hate to have to fork NS for personal use over this.

Time to point out viewtopic.php?f=10&t=20983 here as well I think.

EDIT Sent Giorgio a PM to bring this to his attention, it would be really bad if these issues land in 2.6.9.30 release.

[barbaz]

Giorgio has fixed the problems mentioned in this thread, in NoScript 2.6.9.30rc5. I'm marking this resolved

[GuestPassingBy]

I saw Giorgio's posts in viewtopic.php?f=10&t=20965 and 2.6.9.30rc5. Since I'm guest and can only post here:

When I set allowWhitelistUpdates to false, my objective was to avoid both additions and removals. I know several others who set it to false, and each of them did so with the same purpose (the subject was discussed during a tech chat). I think the type of people who would learn of the pref and consider changing it would be most likely to interpret "updates" as being "additions and/or removals". In addition, that is how the code used to work. So, I don't agree with the new interpretation and different handling. Not to replace my earlier suggestion of GUI controls, but I'd like to mention these options:

1. Explicitly note this change in release notes. Most don't read, future users can misinterpret pref name.
2. Change the pref name to allowWhitelistAdditions and note the change. Better communicates the pref's behavior.
3. Change the pref name and type in order to support >2 options (NoUpdates, RemovalsOnly, RemovalsAndAdditions) and note the change. Better end-user control.

[GuestPassingBy]

Thank you, btw. To you and Giorgio.

[barbaz]

The problem with letting the user disable removals is that if a domain covered by the default whitelist somehow ends up in malicious hands (or is vulnerable), then NoScript would have a security hole caused by NoScript and Giorgio couldn't push an update to close it, thus making NoScript not truly secure (and in a non-obvious way at that) for those users. Plus what if that pref gets set that way accidentally without the user realizing (that sort of thing has happened to me on a few occasions)?

+1 to the rest of your suggestions though.

Thank you, btw. To you and Giorgio.

On behalf of both of us, you're welcome.

[GuestPassingBy]

if a domain covered by the default whitelist somehow ends up in malicious hands (or is vulnerable), then NoScript would have a security hole caused by NoScript

Yes. However, that issue doesn't really apply to those who maintain and take responsibility for their own whitelist. I mean, the removal can still be applied to noscript.default (in case such users reset), but there is no need to perform such a removal where the user has made their own decision to allow a site to be on their whitelist.

I'm thinking that NoScript doesn't have a way to reliably determine whether an entry is "user owned" or "developer owned". If the default whitelist and updates were treated more like a subscription that can be added/removed, intent would be clear. I'm not sure how well that would work out from a usability POV though ("user rule" vs "subscription rule" conflict resolution for example). At present, I think you'd need a separate mechanism for declaring responsibility for the whitelist. We have one though: the control or pref used to control updates! If a user explicitly opts-out of all updates including removals, clearly they are taking ownership of the whitelist and NoScript/Giorgio can't be blamed.

That's just a comment I can think of and one that I think would be good to make. I, personally, am slightly concerned about a developer pushed removal of something I want to be on the whitelist, but I'm not very concerned. None of these updates, so far at least, have applied to my setup. I've forgotten a few times, but I normally do read the changelog and sometimes check source before updating.

[Thrawn]

I'm thinking that NoScript doesn't have a way to reliably determine whether an entry is "user owned" or "developer owned".

I have a suggestion: If you really want to take control of your whitelist and opt out of automatic updates, then wipe the whole default whitelist, clear noscript.default, and repopulate the whitelist with full addresses instead of base second-level domains.

NoScript won't override it, and as a side benefit, you'll be able to whitelist just the HTTPS versions of sites.

[barbaz]

NoScript won't override it,

I don't see where it depends on noscript.default for the whitelist updates...

[GuestPassingBy]

I wanted to try a few things before commenting on Thrawn's suggestion. In the process I may have zeroed in on something else. Can we take a quick sidetrip?

I'm using these for testing, and they pass several checks...

FirefoxPortable_38.0.5_English.paf.exe
SHA256:C12F0D40F3AF3C5CDAC4B552BB6267CA732961BB88AD3E51B56EB921E677B8D4

noscript_security_suite-2.6.9.29-sm+fn+fx.xpi
6C6A21A12E108C3211D14BD2761E1D43C6E7E9A2D0A28F54E87182C12642D57E

I install that FirefoxPortable. Then I install that NoScript 2.6.9.29 via drag/drop. After the restart, noscript.version appears as a user-set empty string in about config. Unlike in my normal, non-portable, FF 38.0.5, where I have the exact same XPI installed and noscript.version shows 2.6.9.29. I think noscript.version being set to an empty string is what was/is preventing onVersionChanged from being called when I update FirefoxPortable's NoScript to 2.6.9.30rc5 in order to see how the whitelist updates behave.

Does this ring a bell with anyone? Is there a known issue with FirefoxPortable? Anyone feel like trying to duplicate the problem? FWIW, I also tried installing NoScript 2.6.9.39rc5 first and again noscript.version was an empty string afterwards.

[GuestPassingBy]

I hadn't intended to, but I stayed up and did some more work on this. I saw the same noscript.version empty string behavior under other scenarios including a non-portable. I looked at the code a bit more. In noscriptService.js, shortly after the this.setpref("version", ver), there is a ns.setPref("version", "") which is associated with an IOS.newChannel("https://noscript.net/-, null, null).asyncOpen call. Too tired to tell for sure, but it looked like noscript.version might get cleared if noscript.net couldn't be looked up or connected to.

These tests I've talked about were all done WITHOUT an Internet connection. I don't like going online without my armor on, and I don't feel like putting it all on just to run simple tests.

So, I host file'd noscript.net to 127.0.0.1 and started a localhost server. Then I tried it again... install FirefoxPortable, install NoScript 2.6.9.29, check noscript.version afterwards. This time, it was set to 2.6.9.29.

Sleepy brain says: I think there is an issue if NoScript is installed when there is no Internet connection.