[RESOLVED] Surrogate functionality with ABE

Proposals for new surrogate scripts, updates/bug fixes to existing ones, tips and tricks to work around the lazy web.
Post Reply
wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

[RESOLVED] Surrogate functionality with ABE

Post by wxman1 » Wed Dec 09, 2015 5:08 am

As I understand it surrogates kick in when there's any match to a source. ABE kicks in when there's particular action(s) is(are) allowed by origin resource by site, i.e., destination resource. In that vernacular, site and source are congruent between ABE and NS surrogate syntax.

If I have a surrogate for a source, and ABE firewall rule that allows a site resource for one and only one predicate resource, will the surrogate only apply for the one predicate resource - and not apply for all others - or on the basis of the surrogate's existence rendering the ABE firewall ruleset moot?
Last edited by wxman1 on Fri Dec 11, 2015 7:05 am, edited 1 time in total.
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

User avatar
Thrawn
Senior Member
Posts: 3106
Joined: Mon Jan 16, 2012 3:46 am
Location: Australia
Contact:

Re: Surrogate functionality with ABE

Post by Thrawn » Wed Dec 09, 2015 6:28 am

Er...I think you might misunderstand what these tools are doing.

ABE doesn't kick into action to allow things, it takes action to block things. The primary point of ABE is to block dangerous requests such as CSRF.

And surrogates, by default, will run in place of the original script when the script is blocked. If the script is allowed, ordinary surrogates don't run. The primary point of surrogates is to un-break sites when their scripts are blocked, by simulating the existence of the original script.

A matching surrogate will run if ABE has blocked the original script.

Does that clarify anything?
======
Thrawn
------------
Religion is not the opium of the masses. Daily life is the opium of the masses.

True religion, which dares to acknowledge death and challenge the way we live, is an attempt to wake up.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0

wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

Re: Surrogate functionality with ABE

Post by wxman1 » Wed Dec 09, 2015 9:21 am

Thrawn wrote:ABE doesn't kick into action to allow things, it takes action to block things. The primary point of ABE is to block dangerous requests such as CSRF...


Perhaps its purely semantics, but this allows scripts hosted at ssl.google-analytics.com to be invoked at one, and only one, web-site:

Site ssl.google-analytics.com
Accept from .comodo.com
deny


For that ABE ruleset to be of affect ssl.google-analytics.com must be allowed - globally - in NS. What makes it implicitly restrictive is the explicit permission for only one URL. If the SITE - surrogate source - is denied in NS, then surrogate replacement will occur - if a surrogate exists - for any URL invoking ssl.google-analytics.com, and ABE won't fire. The above ABE will permit google-analytic server access by only .comodo.com, and for all other URLs the default NS surrogate will be in affect, if, and only if, ssl.google-analytics.com is allowed.

Thrawn wrote:...And surrogates, by default, will run in place of the original script when the script is blocked. If the script is allowed, ordinary surrogates don't run...

A matching surrogate will run if ABE has blocked the original script.


This is the salient point: if a source is allowed in NS, then no surrogate replacement occurs. If the source is blocked surrogate replacement is implemented. A source must be allowed by NS for ABE to be affective. For any SITE stipulated by an ABE ruleset having predicate of ACCEPT action for any arbitrary predicate resource, no surrogate replacement occurs, for all others - akin to try / catch code - DENY, i.e., block and surrogate replacement occurs.

Perhaps there's a way to specify local hosted JS to be invoked in the ABE ruleset for any ACCEPT predicate resource, and allow the default NS surrogate to be in affect otherwise, i.e., per the DENY, even so the SITE must be allowed in NS for the ABE ruleset to even fire?
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

barbaz
Senior Member
Posts: 9093
Joined: Sat Aug 03, 2013 5:45 pm

Re: Surrogate functionality with ABE

Post by barbaz » Wed Dec 09, 2015 10:02 pm

wxman1 wrote:For that ABE ruleset to be of affect ssl.google-analytics.com must be allowed - globally - in NS.

Nope, it'll be effective either way because ABE is broader in scope than per-site permissions. I've explained this to another user with the same confusion here: viewtopic.php?f=23&t=21401
Hope it helps.

wxman1 wrote:Perhaps there's a way to specify local hosted JS to be invoked in the ABE ruleset for any ACCEPT predicate resource, and allow the default NS surrogate to be in affect otherwise, i.e., per the DENY, even so the SITE must be allowed in NS for the ABE ruleset to even fire?

I think this is called a proxy ;)

It's beyond the scope of NoScript.
*Always* check the changelogs BEFORE updating that important software!
-

User avatar
Thrawn
Senior Member
Posts: 3106
Joined: Mon Jan 16, 2012 3:46 am
Location: Australia
Contact:

Re: Surrogate functionality with ABE

Post by Thrawn » Wed Dec 09, 2015 11:54 pm

wxman1 wrote:Perhaps its purely semantics, but this allows scripts hosted at ssl.google-analytics.com to be invoked at one, and only one, web-site:

Code: Select all

Site ssl.google-analytics.com
Accept from .comodo.com
deny


Yes, that rule will be effective for restricting Google Analytics to one site.

For that ABE ruleset to be of affect ssl.google-analytics.com must be allowed - globally - in NS.

Well, yes, otherwise GA will be blocked everywhere. ABE won't block it on comodo.com, but the regular script-blocking will.

(And allowing in NS is *always* global.)

If the SITE - surrogate source - is denied in NS, then surrogate replacement will occur - if a surrogate exists - for any URL invoking ssl.google-analytics.com, and ABE won't fire.

Yes.

The above ABE will permit google-analytic server access by only .comodo.com, and for all other URLs the default NS surrogate will be in affect, if, and only if, ssl.google-analytics.com is allowed.

Yes. If you don't allow GA, then the surrogate will be in effect everywhere, including Comodo.

Thrawn wrote:...And surrogates, by default, will run in place of the original script when the script is blocked. If the script is allowed, ordinary surrogates don't run...

A matching surrogate will run if ABE has blocked the original script.


This is the salient point: if a source is allowed in NS, then no surrogate replacement occurs.

Not quite true. If something else - eg ABE, Adblock Plus, RequestPolicy, uBlock - blocks the real script, then the surrogate will still run.

For any SITE stipulated by an ABE ruleset having predicate of ACCEPT action for any arbitrary predicate resource, no surrogate replacement occurs, for all others - akin to try / catch code - DENY, i.e., block and surrogate replacement occurs.

Isn't that the expected/desired behavior? Why are you allowing Google Analytics on Comodo if you don't want the real GA to run?

Perhaps there's a way to specify local hosted JS to be invoked in the ABE ruleset for any ACCEPT predicate resource, and allow the default NS surrogate to be in affect otherwise, i.e., per the DENY, even so the SITE must be allowed in NS for the ABE ruleset to even fire?

Well, as documented on Giorgio's blog, you can define surrogates that run after whitelisted scripts. But are you sure this is what you want?

What is your actual objective?

Are you trying to run the real Google Analytics on comodo.com, but nowhere else? Then just use the ABE rule you mentioned, whitelist ssl.google-analytics.com, and you're done.
Are you trying to replace Google Analytics with a surrogate on comodo.com? Then you don't need ABE, just leave google-analytics.com blocked, as it is by default.
Are you trying to use a different Google Analytics surrogate on comodo.com to the surrogate used everywhere else? You don't need ABE, just leave GA blocked, then write your new surrogate and specify the correct 'sources' value.

If I knew what you actually want to achieve, I would be better able to help.
======
Thrawn
------------
Religion is not the opium of the masses. Daily life is the opium of the masses.

True religion, which dares to acknowledge death and challenge the way we live, is an attempt to wake up.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0

wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

Re: Surrogate functionality with ABE

Post by wxman1 » Thu Dec 10, 2015 5:05 pm

You almost had it with this:

Are you trying to use a different Google Analytics surrogate on comodo.com to the surrogate used everywhere else? You don't need ABE, just leave GA blocked, then write your new surrogate and specify the correct 'sources' value.


In an ideal world, .comodo.com can invoke full feature google-analytics scripts hosted locally, all other URL implement NS default surrogate for google-analytics. There are two issues at play in my mind: the scripts themselves, and where specifically they live.

Perhaps this is a moot point; I misunderstand the threat. On the basis that many sites are hosting their own jQuery - apparently because of an increasing distrust of Google itself and not jQuery inherently - the script in and of itself is benign. As far as I know, Google-analytics is a site traffic statistics data-miner. I don't care if my surfing habits are tracked on certain sites. In those cases, I would permit webmasters, of those sites only, access to whatever information scipts hosted at google-analytics can provide them. For all other sites, I'm perfectly happy if they can avail themselves to whatever data can be gleaned through implementation of the default NS Googgle-analytics surrogate.

I guess my naivete depends on whether google-analytics based scripts are inherently nefarious, or only on the basis that such scripts are served by Google, or both.

As it stands with the existing config, .comodo.com has permissions to invoke scripts hosted at google.analytics, all other URLs will implement default NS surrogate. Local hosting notwithstanding, the only vulnerability I see is accessing the google-analytics servers themselves - obviously for .comodo.com only. I'm unclear how great a vulnerability access to google-analytics is in and of itself, or if the more serious issue is the scripts themselves regardless where they're hosted; nobody can be trusted whatsoever with the data that can be mined thereof.
Last edited by wxman1 on Thu Dec 10, 2015 9:51 pm, edited 1 time in total.
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

barbaz
Senior Member
Posts: 9093
Joined: Sat Aug 03, 2013 5:45 pm

Re: Surrogate functionality with ABE

Post by barbaz » Thu Dec 10, 2015 8:14 pm

wxman1 wrote:In an ideal world, .comodo.com can invoke full feature google-analytics scripts hosted locally,

Image Why host it locally if you want the full featured script?

wxman1 wrote:There are two issues at play in my mind: (1) the scripts themselves, and (2) where specifically they live.

(numbering mine)
Issue (1) is a very real issue and the only one you should be concerned about here. (2) doesn't apply here because the hosting site is the official site.
(In the cases where the host is not the official site, is some 3rd-party, (2) is simply a matter of whether you personally trust the host or not, no one can help you determine that.)

wxman1 wrote:On the basis that many sites are hosting their own jQuery - apparently because of an increasing distrust of Google itself and not jQuery inherently - the script in and of itself is benign.

Meh. Can also pull jQuery from jQuery's own official CDN site or Microsoft's CDN. I don't think it's anything to do with Google distrust it's probably more like to do with the sites wanting reliability (if the site is online & requires jquery, jquery *will* be available, no need to depend on someone else).

What on that basis?

wxman1 wrote:I guess my naivete depends on whether google-analytics based scripts are inherently nefarious, or only on the basis that such scripts are served by Google, or both.

Neither G-A scripts nor the G-A site are at all nefarious to people who want to be tracked and have their data mined.

wxman1 wrote:I'm unclear how great a vulnerability access to google-analytics is in and of itself, or if the more serious issue is the scripts themselves regardless where they're hosted; nobody can be trusted whatsoever with the data that can be mined thereof.

Access to a server is not a "vulnerability" in and of itself, ever. Again, the scripts themselves are all you need to be worried about here (if even that).
*Always* check the changelogs BEFORE updating that important software!
-

User avatar
Thrawn
Senior Member
Posts: 3106
Joined: Mon Jan 16, 2012 3:46 am
Location: Australia
Contact:

Re: Surrogate functionality with ABE

Post by Thrawn » Thu Dec 10, 2015 10:03 pm

Ah, looks like we're making some progress here.

However, locally hosting jQuery is not the same as locally hosting Google Analytics.
The point of jQuery is to provide extra client-side functions that the page finds useful, like searching and sorting tables.
The point of Google Analytics is to enable tracking and reporting to the server.

So, a locally-hosted, completely-client-side jQuery makes sense, but a completely-local Google Analytics does not make sense.
======
Thrawn
------------
Religion is not the opium of the masses. Daily life is the opium of the masses.

True religion, which dares to acknowledge death and challenge the way we live, is an attempt to wake up.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0

wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

Re: Surrogate functionality with ABE

Post by wxman1 » Thu Dec 10, 2015 10:08 pm

Thanx for the feedbag. Turns out it was an entirely moot issue; google-analytics was being blocked in my HOSTS file from MVPS. :oops:

by disabling that, I then allowed it in NS globally and created an ABE rule:

Site wwwDOTgoogle-analyticsDOTcom
Accept from No_Such_Site.com
deny
#


Then I implemented the following surrogate preferences in about:config:

noscript.surrogate.urchin.sources wwwDOTgoogle-analyticsDOTcom/urchin.js
noscript.surrogate.urchin.replacement file://C:%APPDATA%/Comodo/IceDragon/Profiles/NS_Scripts/urchin.js

This will specifically only open the door for urchin.js. I'll have to get a handle on just how MUCH access to urchin.js specifically is being given carte blanche. :|

EDIT: but that may all be a moot issue again; as has been mentioned that access to google servers in and of itself is not nehparious - its the scripts they're hosting that are - and only in so far that tracking is inherently nefarious. For certain specific sites, I don't mind if they avail themselves of fundamental statistical functionality that can be garnered by certain scripts that the google-analytics service provides to webmasters.
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

Re: Surrogate functionality with ABE

Post by wxman1 » Thu Dec 10, 2015 11:19 pm

checking into another trusted site's implementation of Google-Analytics precipitated my one-post-wonder here: viewtopic.php?f=7&t=21477

This compelled me to change the ABE rule to this:

Site wwwDOTgoogle-analytics.com
Accept from .cod.edu
deny

The surrogate for urchin.js holds - based on the deny - and .cod.ude get their web-site metrics despite different scripts running. Heh. Given that urchin.js is deprecated it prolly won't be that big of a deal.
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

User avatar
Thrawn
Senior Member
Posts: 3106
Joined: Mon Jan 16, 2012 3:46 am
Location: Australia
Contact:

Re: Surrogate functionality with ABE

Post by Thrawn » Fri Dec 11, 2015 1:47 am

Why are you using surrogates to host the Urchin file locally? If you're going to allow those sites to track you via Google, why not just apply the ABE rule, allow GA, and leave it at that?
======
Thrawn
------------
Religion is not the opium of the masses. Daily life is the opium of the masses.

True religion, which dares to acknowledge death and challenge the way we live, is an attempt to wake up.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0

wxman1
Junior Member
Posts: 44
Joined: Tue Dec 08, 2015 8:11 pm

Re: Surrogate functionality with ABE

Post by wxman1 » Fri Dec 11, 2015 7:05 am

Yep. Not only because there's little benefit, I determined that some of the trusted sites are implementing analytics.JS, or GA.JS, and the mechanism employed is dynamic, i.e., at page load and per parameters; I'm not going to maintain variant surrogates.

So I have a handful of sites I'll maintain an ABE rule for so as to allow them explicitely the ability to generate traffic statistics on page usage; and the rest will have to get on the best they can with the default ga.JS NS surrogate having source *.google-analytics.com.

If a site breaks because the local hosted jQuery, jQuery-UI, or default surrogate ga.JS fails to provide the necessary funcions, I know how to determine what's being invoked and how to accommodate that. Furthermore, if the ga.JS is invoked as a 'load' function implementation, I'm aware there's a method to accommodate that situation also.

8-)
Mozilla/5.0 (Windows NT 5.2; rv:42.0) Gecko/20100101 Firefox/42.0

Post Reply