"Allow sites to push their own rulesets" causes crashes

[RobertDrew]

"Allow sites to push their own rulesets" causes Firefox to crash on certain websites.

I've been trying for the past few days to determine exact steps to reproduce, but I have been unsuccessful. I suspect there is something like a race condition occurring, because of the nature of the results I've observed ...

I have two separate Firefox profiles which can reproduce the issue near 100% of the time on particular websites. In these profiles, I can prevent the issue (at or near 100% of the time) on all 3 sites I've taken note of being affected by unchecking "Allow sites to push their own rulesets". Alternatively, I can prevent the issue on 2 of the 3 sites by disabling HTTPS Everywhere.

The affected sites include runnersworld.com and two other sites which are not public.

Because I have not been able to reproduce in a fresh profile, I have attempted to narrow down the variables involved both on the profile side and the website side.

On the website side (for the two non-public sites), I have tried to determine what page element(s) are triggering the issue by removing things one by one on a test page. I have not been successful in identifying exactly what is triggering the issue because the results become inconsistent, though I've noticed that the more things I delete from the page, the less frequently the issue occurs ... down to under 20% (remember that the issue occurs near 100% of the time before I start taking things out of the page).

On the profile side, I have tried to determine what extensions or settings might be involved in triggering the issue, but again I have not been successful because the results become inconsistent. I can still reproduce the issue near 100% of the time when all extensions other than NoScript have been disabled for 1 of the 3 sites. For all 3 of the sites, I can still reproduce the issue near 100% of the time when all extensions other than NoScript and HTTPS Everywhere have been disabled. Once I start removing customizations from my prefs.js and deleting other profile folder files, the frequency of the crashes decreases, again down to under 20%.

These results lead me to believe there may be something like a race condition which is preventing consistent reproduction of the issue which appears to me to originate in the "Allow sites to push their own rulesets" code.

BTW, I have Firefox set to clear my browser history on exit; I noticed that, once an affected page has made it into the browser cache (in an instance where it doesn't crash), it doesn't seem to cause crashes on subsequent navigation to the same page until I re-open Firefox.

Help!

EDIT:
Occurs with just the following changes from default NoScript configuration:
- check "Scripts Globally Allowed (dangerous)"
- check "Allow sites to push their own rulesets"

EDIT 2:
Also note that I have a very slow Internet connection (1 Mbps), which may be relevant.

EDIT 3:
I have now been able to reproduce in a fresh profile with just NoScript and HTTPS Everywhere installed, but only sporadically (maybe 10% of the time). Keep in mind that HTTPS Everywhere is not strictly necessary to reproduce this bug; it seems that the bug is simply more likely to be triggered when the browser is doing more things.

Updated list of sites known to be affected:
runnersworld.com
sears.com
googleonlinesecurity.blogspot.com
addons.prestashop.com

[barbaz]

1) What are the crash report URLs from about:crashes (post in url or code tags please because this forum has a bug where it'll truncate plain links)

2) Can you create a .tar.gz archive of an affected profile (archived when Fx is completely shut down)? As in, you unarchive the profile and the issue will still reproduce (at least for you). Given that there seem to be a number of possibly profile-specific variables, that might be the easiest way to confirm whether this is a NoScript bug or an existing bug of which NoScript is just the messenger.
Probably best to send it to Giorgio Maone, since he's the developer and he uses Windows; if you instead PM me a link I'll post it in the staff-only section of the board where Giorgio & other Moderators can look at it, as I don't use Windows I might not be able to reproduce it.

3) Are you able to send Giorgio any of the non-public page(s) for him to put on one of his private servers (if he asks for it) for testing?

[RobertDrew]

Note: in cases where the page begins to render before crashing, certain page elements which had been rendered disappear shortly before Firefox ultimately crashes.

With the profile which I will send to you (used on a Linux system, though mozilla profiles are compatible across systems), I was able to reproduce the issue 100% of the time with runnersworld.com. When unchecking "Allow sites to push their own rulesets", the problem occurred 0% of the time with runnersworld.com. When disabling HTTPS Everywhere, the problem occurred 0% of the time with runnersworld.com.

I will also send a copy of the runnersworld.com home page as it exists as of shortly before this posting. I am including the full web page version. I am able to reproduce using the archived home page. This eliminates possible issues on the server side, and eliminates the possibility that daily changes to the site may impact the ability to reproduce.

Here's a crash report URL for you: https://crash-stats.mozilla.com/report/ ... 0e22150618

Yes, I am able to reproduce after unarchiving the copy of the profile which I will be sending.

If I notice any other public sites which are affected, I will make note of them in this thread.

[barbaz]

Thanks for the profile & homepage, I'll post those links in the Mods forum for now and do some investigating on my Fx test machine (Linux Mint KDE) later.

Here's a crash report URL for you: https://crash-stats.mozilla.com/report/ ... 0e22150618

That's an odd crash report. Is this a self built (or otherwise 3rd-party) Firefox?

[barbaz]

I can reproduce the issue with that profile on my Firefox test machine, with Firefox 38.0.5.
I just replaced my ~/.mozilla with the one in the .tar.gz and tried to go to runnersworld.com, and it hung for a bit then crashed.

Here's a more informative crash report: https://crash-stats.mozilla.com/report/ ... 5722150618

(Something that might be helpful is net traffic logs, but I don't have time to get that now. I'll try later if someone else hasn't posted that already.)

[RobertDrew]

That's an odd crash report. Is this a self built (or otherwise 3rd-party) Firefox?

It was packaged by "stransky" of the Fedora Project. My guess is that the oddities you are referring to are result of the fact that this crash took place within a virtualized environment.

[barbaz]

It was packaged by "stransky" of the Fedora Project. My guess is that the oddities you are referring to are result of the fact that this crash took place within a virtualized environment.

Yep, that explains why your crash report had so little information. The crash-stats server only works with official Mozilla builds, and apparently stransky does his/her own builds to package for Fedora.

The oddities I was referring to were the lack of debug symbol information about internal Firefox binaries and shared libraries, it's nothing to do with you being in a virtual environment.

[RobertDrew]

addons.prestashop.com also seems to be affected, although I can't reproduce as consistently. Seem to be able to reproduce about half the time on average.

[barbaz]

I sent RobertDrew & forum staff a screenshot of the Browser Console messages before the crash (with net logging enabled), and the only thing that stands out is that there is no response from https://i.yldbt.com/rules.abe .
Going there manually gives a blank page (no content), using HTTPFox on that request (in my normal browser profile) gives the following

Code: Select all

(Request-Line)	GET /rules.abe HTTP/1.1
Host	i.yldbt.com
User-Agent	Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/38.0.2125.111 Chrome/38.0.2125.111 Safari/537.36
Accept	text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language	en-US,en;q=0.5
Accept-Encoding	gzip, deflate
DNT	1
Connection	keep-alive

Code: Select all

(Status-Line)	HTTP/1.1 404 NOT FOUND
Server	nginx/1.6.2
Date	Tue, 23 Jun 2015 16:12:21 GMT
Content-Type	text/html; charset=utf-8
Transfer-Encoding	chunked
Connection	close
Content-Encoding	gzip

[RobertDrew]

Affects sears.com as well.

[RobertDrew]

Also affects googleonlinesecurity.blogspot.com ...

Is there any reason for me to continue to make note of any other sites I notice are affected at this point?

[barbaz]

Is there any reason for me to continue to make note of any other sites I notice are affected at this point?

It could be useful for looking for what all these sites may have in common (if anything). Also it's less likely that we'll lose all the available testcases (if affected site(s) change such that it's no longer affected - if it's not reproducible anymore on one site, others to test against will already be noted).

But so as not to clutter the topic, from here on you might want to consider editing them in to a single post (how about using the one that currently reads "Affects sears.com as well."?) rather than making a separate post for each site (unless it's been at least 2 weeks since any reply and you want to bump this thread and the "Bump topic" link isn't available).

[RobertDrew]

I've done a bit of research to get a general idea about adoption of ABE for Web Authors as I've been reconsidering keeping "Allow sites to push their own rulesets" enabled due to the stability issues noted in this thread.

I checked the status codes for Alexa's top 500 sites (for both global and US), and it appears that none of them have adopted ABE for Web Authors. (Caveat: Alexa doesn't distinguish subdomains, so it's possible that some subdomains of the top 500 sites have in fact adopted ABE.)

There was one interesting result, however, for fidelity.com:

# For future use with http://noscript.net/abe/index.html

Under the circumstances, I have decided to disable "Allow sites to push their own rulesets" in my profile.

It is my suggestion that the option be removed from the preferences GUI and only be accessible via about:config

[barbaz]

viewtopic.php?p=80919#p80919

Was trying to open this page on "The Atlantic", and Firefox crashed... http://www.theatlantic.com/health/archi ... ly/433863/

Tried again several more times, but the page kept crashing Firefox.

[...]
This is the only page I can get the crash on. Tried many other pages at The Atlantic, but all others worked well.

viewtopic.php?p=80921#p80921

https://crash-stats.mozilla.com/report/ ... 81b2160129
[...]
I tried a clean Firefox profile, with a default install of NoScript.

Now the page did not crash Firefox using the clean profile.

Looked to see if anything was set differently in my normal Firefox profile, as far as NoScript settings go. I use a mostly default config.

Found one ABE setting, "Allow sites to push their own rulesets", that was enabled. It was disabled in the clean default profile.

Unchecked it and my normal Firefox works ok on this web page now.