InformAction Forums

Posted: **Mon Jul 24, 2017 2:13 am**

Thanks PLD for discovering this.

Firefox 54.0
NoScript 5.0.7.1rc1
new profile

STR:
1) NoScript Options > Advanced > ABE > USER, add

Code: Select all

Site ALL
Deny INC(OTHER)

2) try to browse to https://noscript.net/

Expected results: site loads and looks normal

Actual results: site is totally blocked by ABE. Related console messages -

Code: Select all

[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/ <<< chrome://browser/content/browser.xul - 1}
USER rule:
Site ALL
Deny INCLUSION(OTHER)

So I tried changing the ABE rule to -

Code: Select all

Site ALL
Accept from chrome: moz-nullprincipal:
Deny INC(OTHER)

While the NoScript page now loads, it is quite badly messed up. New related console messages -

Code: Select all

[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/oss.css <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/iasw.jpg <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://api.flattr.com/button/flattr-badge-large.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/noscript/logo.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/noscript/ss0.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net//images/sponsored-by-nlnet.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/wc06.jpg <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net//badge-flashgot.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/badge-easyspeedy.png <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/donate2.gif <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)
[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/noscript.ico <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)

Disabling e10s seems to get things back to normal. Console output for comparison -

Code: Select all

[ABE] < ALL> Deny INCLUSION(OTHER) on {GET https://noscript.net/noscript.ico <<< https://noscript.net/ - 1}
USER rule:
Site ALL
Accept from chrome: moz-nullprincipal:
Deny INCLUSION(OTHER)

Posted: **Tue Jul 25, 2017 12:29 pm**

Please check latest development build 5.0.8rc2, thanks.

Posted: **Tue Jul 25, 2017 4:26 pm**

Much better, but now Deny INC(OTHER) no longer blocks Websocket connections. This both with and without e10s enabled.

Test page: https://www.websocket.org/echo.html

EDIT
Here's the console message produced for plain Deny INC, NoScript 5.0.8rc2, SeaMonkey -

Code: Select all

[ABE] < echo.websocket.org> Deny INCLUSION on {GET http://echo.websocket.org/?encoding=text <<< http://www.websocket.org/echo.html - 16}
TEST rule:
Site echo.websocket.org
Deny INCLUSION

Console output from 5.0.8rc1, same ABE rule -

Code: Select all

[ABE] < echo.websocket.org> Deny INCLUSION on {GET http://echo.websocket.org/?encoding=text <<< http://echo.websocket.org/?encoding=text, http://www.websocket.org/echo.html - 1}
TEST rule:
Site echo.websocket.org
Deny INCLUSION

Posted: **Tue Jul 25, 2017 7:20 pm**

@barbaz: Thank you for recognizing the INC issue and starting this thread.
@giorgio: Thanks for 5.08rc2

barbaz wrote:Much better, but now Deny INC(OTHER) no longer blocks Websocket connections. This both with and without e10s enabled.

That would be a step in the right direction given that it is its own type:

Chrome webRequest.ResourceType:

Code: Select all

"main_frame", "sub_frame", "stylesheet", "script", "image", "font", "object", "xmlhttprequest", "ping", "csp_report", "media", "websocket", or "other"

Firefox webRequest.ResourceType:

Code: Select all

"main_frame", "sub_frame", "stylesheet", "script", "image", "object", "object_subrequest", "xmlhttprequest", "xbl", "xslt", "ping", "beacon", "xml_dtd", "font", "media", "websocket", "csp_report", "imageset", "web_manifest", "other"

But ABE would have to recognize it. I did try Deny INC(WEBSOCKET) and the latest rc didn't accept it.

I'm still testing 5.0.8rc2 INC handling and have two questions. Given this:

Code: Select all

Site .td.test
Deny INC(SCRIPT)

Q1: If a page includes an iframe from test.td.test and that iframe has inline script, should that script execute?
Q2: If a page includes an iframe from test.td.test and that iframe includes script from some.other.domain (that is allowed), should that script execute?

I'm thinking that Deny INC(SCRIPT) is not supposed to cascade to those items, but would like to confirm. Thanks.

Posted: **Tue Jul 25, 2017 9:31 pm**

PLD wrote:@barbaz: Thank you for recognizing the INC issue and starting this thread.
@giorgio: Thanks for 5.08rc2

barbaz wrote:Much better, but now Deny INC(OTHER) no longer blocks Websocket connections. This both with and without e10s enabled.
That would be a step in the right direction given that it is its own type:

It's actually a problem of balancing backward compatibility (some types are relatively recent) and usefulness in current Firefox versions.

Therefore in RC3 I'm gonna update the current set of recognized INCLUDE(XYZ), with some convenience/compat aliases, as follows (where not otherwise specified in parentheses, the name maps directly to the nsIContentPolicy.TYPE_$name constant):

Code: Select all

BEACON; CSP_REPORT; CSS (TYPE_STYLESHEET); DATAREQUEST; DOCUMENT; DTD; FETCH; FONT; IMAGE (TYPE_IMAGE, TYPE_IMAGESET); IMAGESET; INVALID; MEDIA; OBJ (TYPE_OBJECT, TYPE_OBJECT_SUBREQUEST); OBJECT; OBJECT_SUBREQUEST; OBJSUB (TYPE_OBJECT_SUBREQUEST); OTHER (TYPE_OTHER, TYPE_WEBSOCKET); PING; REFRESH; SCRIPT; STYLESHEET; SUBDOC (TYPE_SUBDOCUMENT); SUBDOCUMENT; UNKNOWN (TYPE_OTHER); WEBSOCKET; WEB_MANIFEST; XBL; XHR (TYPE_XMLHTTPREQUEST, TYPE_DATAREQUEST); XMLHTTPREQUEST; XSLT

As you can see, "OTHER" matches TYPE_OTHER (anything not yet uniquely labeled in nsIContentPolicy) and TYPE_WEBSOCKET for backward compatibility. If you don't want to match TYPE_WEBSOCKET but just TYPE_OTHER, use INC(UNKNOWN). If you want to match TYPE_WEBSOCKET only, use INC(WEBSOCKET).

PLD wrote:
Code: Select all
Site .td.test
Deny INC(SCRIPT)
Q1: If a page includes an iframe from test.td.test and that iframe has inline script, should that script execute?

Nope, INCLUDE doesn't match inline content, just discrete loads (like those triggered by <script src="xyz"></script> elements).

PLD wrote: Q2: If a page includes an iframe from test.td.test and that iframe includes script from some.other.domain (that is allowed), should that script execute?

Nope, see above.

Posted: **Tue Jul 25, 2017 10:31 pm**

Please check latest development build 5.0.8rc3, thanks.

Posted: **Tue Jul 25, 2017 10:58 pm**

Giorgio Maone wrote:
PLD wrote:Therefore in RC3 I'm gonna update the current set of recognized INCLUDE(XYZ), with some convenience/compat aliases <snip>

That is great news! A neat way of handling backward compat too. Looks like the bases are covered except for one thing: Would anyone want to address TYPE_IMAGE without also addressing TYPE_IMAGESET? I haven't thought about it and don't know the answer.

As for my rc2 testing, so far so good except:

Code: Select all

Site ALL
Deny INC(SCRIPT)

Is not blocking an external script file which is included via src scheme of ftp: or file: (the file: schemed inclusion happening when test page is loaded via file: - a page relative load actually). Same applies to a few other INC types I tried. I also tried:

Code: Select all

Site ALL
Deny ALL

Test page loads from file: and both file: and ftp: schemed script/other inclusions happen. I'm aware of Thrawn's 2015 comment about ftp being out of scope, but.

Edited to clarify and now see 5.0.8rc3...

Posted: **Tue Jul 25, 2017 11:42 pm**

Giorgio Maone wrote:Please check latest development build 5.0.8rc3, thanks.

Nice, thanks!

Couple questions -

1) Did OTHER include more types before, specifically BEACON, CSP_REPORT, and WEB_MANIFEST? Not that I've used it for such purpose, just wondering from backwards compatible point of view.

2) Should TYPE_FETCH be included by XHR?

Posted: **Wed Jul 26, 2017 11:19 am**

Saw this in 5.0.8rc3:

Code: Select all

HTTPVERB  : 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE'; (ABE.g line 46)

Tried adding ABE rules with HEAD, TRACE, and OPTIONS (not accepted).

New abe_rules.pdf says:

1.2 Methods
The <method> component of a <predicate> can be any HTTP method (GET, POST, HEAD, PUT, DELETE, TRACE, OPTIONS) with the addition of 3 “pseudo” methods:

MDN lists:

GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH

Posted: **Wed Jul 26, 2017 11:42 am**

That's a grammar regression I had already fixed yesterday, but it's so minor that I didn't deem worth of a RC4 just for that

Posted: **Wed Jul 26, 2017 9:44 pm**

Please check latest development build 5.0.8rc4, thanks.

Posted: **Thu Jul 27, 2017 2:12 am**

Now INC(OTHER) is matching scripts.

Test page: https://noscript.net/changelog

If scripts are allowed, the changelog will be formatted with bolding and colors. Otherwise the changelog is just plain text.

This ABE rule -

Code: Select all

Site .noscript.net
Deny INC(OTHER)

... blocks the script that formats the changelogs -

Code: Select all

[ABE] < .noscript.net> Deny INCLUSION(OTHER) on {GET https://noscript.net/cl.js <<< https://noscript.net/changelog - 2}
TEST rule:
Site .noscript.net
Deny INCLUSION(OTHER)

Posted: **Thu Jul 27, 2017 12:32 pm**

barbaz wrote:Now INC(OTHER) is matching scripts.

Fixed in 5.0.8rc5, thanks.
At this moment (may change in the future, if/when further load types are supported by Gecko) OTHER matches the following:

TYPE_BEACON
TYPE_CSP_REPORT
TYPE_DTD
TYPE_FONT
TYPE_INVALID
TYPE_MEDIA
TYPE_OTHER
TYPE_PING
TYPE_REFRESH
TYPE_WEBSOCKET
TYPE_WEB_MANIFEST
TYPE_XBL
TYPE_XSLT

Posted: **Thu Jul 27, 2017 2:45 pm**

Confirmed fixed, but it's still not quite backwards-compatible. INC(OTHER) didn't match TYPE_MEDIA or TYPE_FONT before.

Test page for media: http://www.quirksmode.org/html5/tests/video.html
Test page for fonts: https://fonts.google.com/

ABE rule:

Code: Select all

Site ALL
Deny INC(OTHER)

Posted: **Thu Jul 27, 2017 3:23 pm**

barbaz wrote:Confirmed fixed, but it's still not quite backwards-compatible. INC(OTHER) didn't match TYPE_MEDIA or TYPE_FONT before.

They were not documented as "static" ABE types, but fair enough: it was actually in the formerly "rigid" grammar, and if DTD and XBL have their own type, surely discriminating MEDIA and FONT is more useful. Sigh.

InformAction Forums

[FIXED]ABE incorrectly thinks most requests are type "other"

[FIXED]ABE incorrectly thinks most requests are type "other"

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot

Re: [e10s] ABE incorrectly thinks most requests are type "ot