Community Forum

James Shaw
James Shaw
Offline
Resolved
0 votes
I'm having a problem with the web proxy + content filter configuration on ClearOS 7.3 Community.
With non-transparent + authentication enabled, I have 4 policies plus the included default policy. The default policy applies to the "allusers" group, the other 4 policies are "admins" for the user group "itadmins", "managers" for the user group "managers", "marketing" for the user group "marketing", and "staff" for the user group "staff".
The problem I'm having is that the default policy overrides all of the other policies I've created because all of the users belong to the "allusers" group and (at least through the webconfig portal) I cannot remove them from the "allusers" group. I've tried limited-to-no filtering on the default policy and then restricting some sites like hotmail.com, facebook.com, slashdot.org, etc on each of the policies I created, but the default policy takes precedence. I've tried blocking those sites on the default policy, and then using the exception list on the policies I've created, also to no avail. When disabling user authentication, the IP exceptions and banned sites works just fine however.
Can anyone please offer me some suggestions?
Friday, May 19 2017, 06:37 PM
Share this post:
Responses (6)
  • Accepted Answer

    James Shaw
    James Shaw
    Offline
    Saturday, May 20 2017, 02:55 PM - #Permalink
    Resolved
    0 votes
    I stopped by the office this morning to do some more testing and try to reply with as much detail as possible.
    Although I have only tried this with Firefox (53.0.2 - configured as "Manual proxy configuration, use this proxy for all protocols" port 8080, IP address of the ClearOS box), I can confirm that the browser is asking me for a login user & password, and if I get it wrong, it pops back up to let me know it's wrong. Once I get the password correct, the results are as follows.
    Also fwiw, between each configuration change on the ClearOS box, I rebooted both the ClearOS box and my Windows 7 laptop. Everytime I tried a different user on my laptop, I rebooted as well. The windows 7 laptop does not have adblock, antivirus, or much on it. It's a base install I setup just for testing things like this.

    The users and groups on the ClearOS box are as follows
    james belongs to itadmins and allusers
    testuser1 belongs to managers and allusers
    testuser2 belongs to marketing and allusers
    testuser3 belongs to staff and allusers

    Test Scenario 1: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "james" (blank password on laptop, not ClearOS) for testing purposes.

    Default Policy, Banned Sites: None
    itadmins Policy, Banned Sites: None
    managers Policy, Banned Sites: slashdot.org
    marketing Policy, Banned Sites: hotmail.com
    staff Policy, Banned Sites: facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser1 from managers: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser2 from marketing: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser3 from staff: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.

    Test Scenario 2: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "james" (blank password on laptop, not ClearOS) for testing purposes.

    Default Policy, Banned Sites: slashdot.org, hotmail.com, facebook.com
    itadmins Policy, Exceptions: slashdot.org, hotmail.com, facebook.com
    managers Policy, Exceptions: hotmail.com, facebook.com
    marketing Policy, Exceptions: slashdot.org, hotmail.com
    staff Policy, Exceptions: slashdot.org, facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser1 from managers: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser2 from marketing: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser3 from staff: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine

    Test Scenario 3: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "james" (with the same password for the "james" account on the ClearOS box) for testing purposes.

    Default Policy, Banned Sites: None
    itadmins Policy, Banned Sites: None
    managers Policy, Banned Sites: slashdot.org
    marketing Policy, Banned Sites: hotmail.com
    staff Policy, Banned Sites: facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser1 from managers: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser2 from marketing: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser3 from staff: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.

    Test Scenario 4: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "james" (with the same password for the "james" account on the ClearOS box) for testing purposes.

    Default Policy, Banned Sites: slashdot.org, hotmail.com, facebook.com
    itadmins Policy, Exceptions: slashdot.org, hotmail.com, facebook.com
    managers Policy, Exceptions: hotmail.com, facebook.com
    marketing Policy, Exceptions: slashdot.org, hotmail.com
    staff Policy, Exceptions: slashdot.org, facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser1 from managers: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser2 from marketing: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser3 from staff: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine

    Test Scenario 5: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "bob" (blank password on laptop, user does not exist on ClearOS) for testing purposes.

    Default Policy, Banned Sites: None
    itadmins Policy, Banned Sites: None
    managers Policy, Banned Sites: slashdot.org
    marketing Policy, Banned Sites: hotmail.com
    staff Policy, Banned Sites: facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser1 from managers: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser2 from marketing: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.
    testuser3 from staff: slashdot.org, facebook.com are accessible, hotmail.com redirects to login.live.com and works just fine.

    Test Scenario 6: (Non-transparent + Authentication)
    Tested from Windows 7 Pro x64, Firefox 53.0.2
    Logged into the Windows 7 computer as "bob" (blank password on laptop, user does not exist on ClearOS) for testing purposes.

    Default Policy, Banned Sites: slashdot.org, hotmail.com, facebook.com
    itadmins Policy, Exceptions: slashdot.org, hotmail.com, facebook.com
    managers Policy, Exceptions: hotmail.com, facebook.com
    marketing Policy, Exceptions: slashdot.org, hotmail.com
    staff Policy, Exceptions: slashdot.org, facebook.com
    Results
    james from itadmins: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser1 from managers: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser2 from marketing: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine
    testuser3 from staff: slashdot.org, facebook.com are inaccessible, hotmail redirects to login.live.com and works fine


    Given the results, it appears as though the default policy is the only one taking effect. If it were an error related to the order/sequence, either the exceptions, or banned sites for each policy would have an effect.

    Although the box is a new installation (not in a production environment yet), I can wipe and reload it to be sure there isn't a pre-existing problem with the ClearOS installation if anyone thinks that will help?
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 19 2017, 08:30 PM - #Permalink
    Resolved
    0 votes
    Here is a snippet from some more documentation:
    If the content filter can derive the username it can make classifications as to which policy to apply based on that username and the group membership of that user. The order of the content filter groups is important. The default policy is the top filter group and it is the one that gets applied both FIRST and LAST. First, if the username is NOT specified and LAST if it the username was specified but didn't match. The policy is a first match first apply policy. If a user belongs to multiple groups, the policy listed in which they first match is the one applied.

    In order for the proxy server to receive a username, the browser must supply it. The only way that this can occur is if the proxy server is specified in the browser settings. There are two methods for applying configuration setting for use with User Authentication:


    https://www.clearos.com/resources/documentation/clearos/content:en_us:kb_o_content_filtering_ins_and_outs
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 19 2017, 08:27 PM - #Permalink
    Resolved
    0 votes
    SO here is how the list will appear in Webconfig:

    Default Policy
    Policy 1
    Policy 2
    Policy 3
    Policy N

    Here is how they are actually applied:

    Policy 1
    Policy 2
    Policy 3
    Policy N
    Default Policy
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 19 2017, 08:25 PM - #Permalink
    Resolved
    0 votes
    The content filter policy list is a bit deceptive since the first thing listed is the default policy. Nick is right that the first match is what is applied. This excludes the default policy which is the policy applied if no other policy is triggered.

    When using authentication with your content filter is it important to pay attention to the squid/access.log file for user authentication hits and the dansguardian/access.log for the group authentication hits.

    If your user authentication is not happening in the squid log then the group won't happen and they will get the last policy, the default one. The allusers is not really important but is useful because the default policy is the very LAST policy to be hit and in this case, they match. But even if your traffic did NOT include a match to allusers it would still apply the default policy. Best to look in the squid log file to see if user authentication is actually happening. So the allusers is a bit of a red herring since it doesn't apply here to the process. If your users are making it all the way to the default policy is it because the system was unable to match then to a particular group in the accounts driver or they didn't authenticate.

    You can actually use different browsers to test this since the different browsers have differing support for the transparent NTLM authentication. IN this case, IE will try NTLM authentication but other browsers like Firefox will not behave like this by default. So if you launch firefox and the proxy does NOT ask for a username and password you can validate that the authentication aspect is not happening at all.
    The reply is currently minimized Show
  • Accepted Answer

    James Shaw
    James Shaw
    Offline
    Friday, May 19 2017, 08:19 PM - #Permalink
    Resolved
    0 votes
    Nick, thanks for the quick response. Unfortunately it's not something that can be modified that I can see. From what I understand from this document, https://www.clearos.com/resources/documentation/clearos/content:en_us:kb_o_clearos_guides_setting_up_content_filter_policies the content filter allows you to stack different rules, however it does this in a pre-defined fashion. If I could remove the default policy (the option to remove my own defined policies is there, however for default you cannot), or if I could remove the users from the allusers group (again, this is not an option in the webconfig interface) I suspect this would work. If there's something I can modify in a conf file or through CLI, I suspect that would work, however I suspect I'm missing something in the webconfig, otherwise I couldn't possibly be the first person to have this problem.
    The reply is currently minimized Show
  • Accepted Answer

    Friday, May 19 2017, 07:55 PM - #Permalink
    Resolved
    0 votes
    I don't use the content filter but I believe it operate like the firewall. The first rule it hits stops all further processing. As a result of that, the allusers rule should be at the bottom of your list so the other rules get hit first.

    What is your rule order?
    The reply is currently minimized Show
Your Reply