[FF 4b08pre] Can ABE rules cause jump in FF mem usage?

[milithruldur]

Good day.

I'm experimenting with NoScript's ABE funtion. I have setup my rules as follows:

Code: Select all

SYSTEM:
# Prevent WAN sites from requesting LAN resources
Site LOCAL
Accept from LOCAL
Deny

USER:
Site ^https?://(\w*\.)*y(ahoo|img)\.com/.*
Accept from ^https?://(\w*\.)*y(ahoo|img)\.com/.*
Anon

Site ^https?://(\w*\.)*google\.com(\.\w{2})?/.*
Accept from ^https?://(\w*\.)*google\.com(\.\w{2})?/.*
Anon

Site ^https?://(\w*\.)*mywot\.(com|net)/.*
Accept from ^https?://(\w*\.)*mywot\.(com|net)/.*
Anon

Site .live.com .msn.com ^https?://(\w*\.)*passport\.(com|net)/.*
Accept from .live.com .msn.com ^https?://(\w*\.)*passport\.(com|net)/.*
Anon

Site .lastpass.com
Accept from .lastpass.com
Anon

# Allows httpauth+cookies to be sent with requests
# from the same base domain, stripping it off otherwise
Site *
Accept from SELF++
Anon

With the USER ABE rules active, I notice that Firefox memory usage jumps very quickly, from say 103,000K to 220,000K of VM Size after several browsing. This is true for both Firefox 4 beta 6 and Minefield, and especially happens when I visit my Windows Live account, where there are multiple cross-domain requests being made between live.com, hotmail.com, msn.com, wlxrs.com, and sometimes passport.net passport.com.

After navigating around Windows Live pages (Hotmail, Messenger, Spaces, etc.), I see this significant jump in memory from the task manager. I also have the error console open to view ABE's actions in realtime, and debug if some functions of the Live pages break as a consequence of the rules I have. This jump in memory does not go down even after closing tabs or clearing all caches.

However, if I have the USER ABE rules deactivated during a fresh browser restart, then memory usage is, in my opinion, within normal limits when navigating in Windows Live. I admit that there is an incremental memory usage as Firefox is used, especially after several hours of web surfing, but should not be so much significant increment in a very short period.

Regarding the regular expressions in my rules, I have so far read that using ( <expr> ) and (?: <expr> ) are two different functions, where the former captures the matched substring as indicated by the <expr>, which can be recalled from an array of elements using the the notation \1 ... \n for n matches, while the latter does not. Could this be the cause of the sudden memory jump? Should the latter be used in favor of the former?

Can anybody share their thoughts on this? Thanks.

[Giorgio Maone]

Regarding the regular expressions in my rules, I have so far read that using ( <expr> ) and (?: <expr> ) are two different functions, where the former captures the matched substring as indicated by the <expr>, which can be recalled from an array of elements using the the notation \1 ... \n for n matches, while the latter does not. Could this be the cause of the sudden memory jump? Should the latter be used in favor of the former?

Yes, preferring non-capturing patterns over capturing ones is definitely advisable, even though it may be not a "magic" solution to what might not be a real problem.
JavaScript (which most extensions including NoScript and a large part of the browser itself is implemented in) is a garbage collected language, and therefore the memory you use is not reclaimed back immediately, but at a later time, possibly when there's actually a low-memory pressure, in order to minimize the performance impact of the garbage collector.

Therefore if you create lots of JavaScript objects (like it happens when you use regexps in general, and capturing patterns especially, almost on every browser request), you quickly raise the memory usage level and, since you keep adding stuff, it may take a long time before the situation stabilizes back to a more "normal" level.

However the GC mechanism will prevent you from going "out of memory", even though the perceived usage may seem disproportionate to other programs.

Getting back to your original question, yes, please convert your expressions to use non-capturing patterns and check whether this improves the situation a bit.

[milithruldur]

Thank you for your informative response.

At first I thought it might have something to do with having the error console open and capturing ABE messages, or perhaps to a certain degree it might add to more memory consumption.

I have updated the regexes to use the non-capturing parentheses, and browsed several pages of the Windows Live site. It seems to have tamed memory consumption. And because of recent changes in Firefox nightlies concerning javascript compartmentalization, it seems to have better improved garbage collection, as I notice that after running a js benchmark on one tab, and closing that tab after, it recovers memory after a couple of seconds to within "normal" values as before the benchmark was started. This is apart from the security enhancements compartmentalization brings to javascript objects.

Will report back if any further oddities happen.