[RESOLVED] NoScript 2.0 and 2.0.1rc1 performance problems

[al_9x]

But isn't normal browsing more than usable, instead?

Not quite, as I mentioned, I have been regularly running into less severe, briefer, locking spikes, with fewer tabs. I didn't pursue it, but this synthetic test in a new profile, shows (to me) there is indeed a NS problem.

As I said, I'm gonna investigate ways to alleviate this edge case on slow machines, it's just not a priority (this week at least) because we lived just well so far

My offer stands. Next week?

If it's not the <META>, it may very well be a JavaScript or a Flash reload, but if you look closely you'll notice that every page there reloads itself just after being loaded the first time.

There is no javascript or flash, per the parameters of the experiment (cnn is not allowed, and all plug-ins are disabled). I begin browsing only when all the loading stops, and the cpu activity goes to and stays at idle. And think carefully about what you're saying, if the page itself is at fault, then it would get worse without NS, instead the prolonged locking spikes happen with NS, but not without it.

[therube]

There is no javascript or flash, per the parameters of the experiment (cnn is not allowed, and all plug-ins are disabled).

Note that I only ran into issues once cnn.com was Allowed.
(And only cnn.com, no other domains - other then those that may be whilelisted from a clean install.)
Even though Apply these restrictions to whitelisted sites too is not defaulted, & even with cnn.com Allowed, it looks like other domains (turner.com) are/may be needed before Flash becomes operational. (Don't know about other plugins.)

[therube]

(Now on a lessor computer, an Intel E4300, 2 GB ram, WinXP rather then W7.)

> 3. I disabled all extensions and plugins.

When I did the test, I glossed over the "plugins" part.
Doing so, 50x cnn.com opened extremely quickly & I have no issues what so ever.

Mem usage went from ~50 MB (simply opening FF) to ~550 MB w/50x - to be expected.
Tabs, scrolling, links, all respond crisply.


Let me Quit, reload, load (1) cnn.com, Allow, Quit, reload, then open 50x. ...

[therube]

Only change is to Allow cnn.com.
Plugins still disabled, no other extensions installed.

Network activity pretty much stopped at 3 min 28 sec.
CPU was running ~50% (1 of two cores) till about 5 min.

FF is less then responsive. Simply clicking a tab may cause 50% CPU spikes.
I would not want to be running like this.

I may catch a time where I can click, click, click, but then CPU jumps & I have to wait for it to reside.

So plugins are not at issue.

For me, a non-allowed domain is not an issue.
(I.e. I have no problems so long as cnn.com is not Allowed.)
Only once the domain (cnn.com) is Allowed do I start to have problems.

[therube]

Revoked cnn.com.

Both network & CPU resided in about 40 seconds.
(By default, all tabs were refreshing.)

Same session still, but with cnn.com Revoked.
No issues click tabs, opening links ... all responding normally.

CPU is nil (except for the expected spike during the actual page event), Mem usage ~675 MB.

[al_9x]

I retested with NS 1.10 in the vm, and it loaded in 3 min 50 sec (3 min 10 sec without NS) That's a dramatic improvement from the 18-19 minutes with 2.0.9.9rc4. The subsequent cpu spikes on browsing were smaller and much shorter and the browser was actually usable.

There is no question here that there has been a major regression in NS performance between 1.10 and now on less powerful CPUs, CPUs which nevertheless can not be dismissed as inadequate for browsing.

[Giorgio Maone]

@al_9x:

Premised that

1. Merely loading the CNN page once breaks Venkman's profiler, which therefore is unusable to spot any potential bottleneck
2. I'm writing an alternate all-JS rudimentary profiler and first measures suggest there's no real bottleneck inside NoScript code, but rather something (memory pressure? GC fragmentation?) causing Firefox's code to run slower
3. I cannot focus on this issue anymore until after this weekend
4. You seem to have found an older build which works better for you

could you please figure out a "regression" range, i.e. the first version which showed these symptoms?

[al_9x]

could you please figure out a "regression" range, i.e. the first version which showed these symptoms

ok.

[al_9x]

could you please figure out a "regression" range, i.e. the first version which showed these symptoms?

Focusing on the loading time, since that's easier to quantify and compare, the regression window is 2.0.1-2.0.2

[Giorgio Maone]

could you please figure out a "regression" range, i.e. the first version which showed these symptoms?

Focusing on the loading time, since that's easier to quantify and compare, the regression window is 2.0.1-2.0.2

Looking at the changelog, I'd say you should try disabling surrogates and toStaticHTML.

[al_9x]

Looking at the changelog, I'd say you should try disabling surrogates and toStaticHTML.

In 2.0.2 that cut it down to 3m40s (about the same as 2.0.1 at defaults) from 12m10s. The time without NS is 2m10s.

I will try the current build later today to see if in addition to the loading times this takes care of of the post loading locking spikes, which is the main issue of this thread.

So what's going on here? Can anything be done about this regression? Even pre regression it's 70% worse with NS, but that's manageable.

[al_9x]

I will try the current build later today to see if in addition to the loading times this takes care of of the post loading locking spikes, which is the main issue of this thread.

It does. There's still some spiking but the browser is usable.

[Giorgio Maone]

I will try the current build later today to see if in addition to the loading times this takes care of of the post loading locking spikes, which is the main issue of this thread./quote]

It does. There's still some spiking but the browser is usable.

I've positively determined that spiking is due to garbage collection taking a insane amounts of time after all those tabs have been opened (probably due to memory fragmentation). In my quite fast system, up to 3 seconds per cycle. That looks definitely like a cycle collector issue (not necessarily a bug in the proper sense, but certainly something they should look into for improvements), which NoScript puts in the spotlight because of the additional data structures that are created and destroyed in order to check each HTTP request or, in this case, to inject surrogates.

The surrogate performance difference between 2.0.1 and 2.0.2 is almost surely due to the fact 2.0.2 and above use sandboxes instead of DOM manipulation in order to execute content scripts. This is much safer and reliable, but apparently comes with a speed cost. On the other hand, this is the very same method used by GreaseMonkey and now by Jetpacks, so this situation may be taken care of in the core (especially because Jetpacks are being promoted as the officially sanctioned way to develop extensions for Firefox 4 and above).

At any rate I'm looking into ways to speed up things on my side.

[al_9x]

I've positively determined that spiking is due to garbage collection taking a insane amounts of time after all those tabs have been opened (probably due to memory fragmentation). In my quite fast system, up to 3 seconds per cycle. That looks definitely like a cycle collector issue (not necessarily a bug in the proper sense, but certainly something they should look into for improvements), which NoScript puts in the spotlight because of the additional data structures that are created and destroyed in order to check each HTTP request or, in this case, to inject surrogates.

So the big difference between 2.0.1 and 2.0.2 with respect to the spikes is understandable? Why doesn't it stop after everything is done and stabilized? Shouldn't those structures be released and collected after a while?

The surrogate performance difference between 2.0.1 and 2.0.2 is almost surely due to the fact 2.0.2 and above use sandboxes instead of DOM manipulation in order to execute content scripts. This is much safer and reliable, but apparently comes with a speed cost. On the other hand, this is the very same method used by GreaseMonkey and now by Jetpacks, so this situation may be taken care of in the core (especially because Jetpacks are being promoted as the officially sanctioned way to develop extensions for Firefox 4 and above).

At any rate I'm looking into ways to speed up things on my side.

Pending those prospective improvements, can you add the 2.0.1 method (off by default)? It would provide immediate relief from this problem.

[Giorgio Maone]

Shouldn't those structures be released and collected after a while?

They're released immediately (as soon as each request is completed) and presumably collected after a while, but in my understanding they merely "happening" and being small things causes the garbage collection process to become slower and slower over the time (looks like some form of memory fragmentation).

Pending those prospective improvements, can you add the 2.0.1 method (off by default)? It would provide immediate relief from this problem.

I'll check whether it's possible.