Forums

Resolved
0 votes
Hello,

I am trying to setup Clearos on Google Cloud Engine. I used as a base the Centos 7 image (x86_64 built on 20180716) and transform it with help of the script mentioned here:
https://www.linode.com/stackscripts/view/13940-clearos-ClearOS+7

I know, that ClearOS uses iptables instead of Firewalld, but for sshd restart there is a process calling Google Compute Engine Instance Setup. It does some synchronization (which is not important now). What is important is, that this process needs to access URL <blockquote>http://metadata.google.internal/</blockquote>, which is translated int 169.254.169.254. Problem is, that when Clearos Firewall Engine is running, this communication is blocked. If I stop the Clearos Firewall, I can communicate with that server.
It looks only this one server is blocked. I tested it with curl and servers e.g. iana.org or google.com provides answer, but the url above just hangs. When Clearos Firewall Engine is switched off, it start to work.
There is no problem with the Firewalld.

I don't use Egress Firewall, all outgoing ports/addresses are allowed. There is only one network interface.
Output from "iptables -nvL -t nat" and "iptables -L " attached. I there rule for 169.254.0.0/16, but it is for incoming communication, not for outgoing.

Can you please help me solve this issue? Any help is appreciated.

Regards
Ondrej
Tuesday, August 14 2018, 04:22 PM
Share this post:

Accepted Answer

Tuesday, August 14 2018, 04:44 PM - #Permalink
Resolved
0 votes
Yes but the incoming block is before the related/established allow so blocks all return traffic :( You'll need another firewall rule to override this one as it is a default rule.Unfortunately it will have to be a custom rule. You can make it specific to the IP:
iptables -I INPUT -s 169.254.169.254 -j ACCEPT
or a generic one to allow all related return packets:
iptables -I INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
Or various combinations.

Try it a the command line first and if it works, make a custom firewall rule changing "iptables" to "$IPTABLES".

I am very surprised anyone is using a 169 address. I don't believe they should be used.
The reply is currently minimized Show
Responses (11)
  • Accepted Answer

    Thursday, August 16 2018, 05:55 AM - #Permalink
    Resolved
    0 votes
    Hi Nick,

    done. Thank you. I tried to solve my situation before I run the initial wizard, so with no apps installed.

    Regards
    Ondrej
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 07:54 PM - #Permalink
    Resolved
    0 votes
    Please remove your "custom" file or move it to one side then install the Custom Firewall app. Then reapply your rule through the Webconfig. I did not realise you had not installed the app.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 06:32 PM - #Permalink
    Resolved
    0 votes
    Hello Nick,

    I have no file called custom in my fresh installation. Just local and types.

    [root@clearos ~]$ cd /etc/clearos/firewall.d/
    [root@clearos firewall.d]$ ll
    total 8
    -rwxr-xr-x. 1 root root 187 May 22 18:45 local
    -rwxr-xr-x. 1 root root 1467 May 22 18:45 types
    [root@clearos firewall.d]$


    I created the file custom with the content you posted here. Here is the final version aligned to your recommendations (I hope;) )


    [root@clearos ~]$ cat /etc/clearos/firewall.d/custom
    #######################################
    # Created by API - Please Do NOT Edit #
    #######################################
    # IPv4 Custom Firewall Rules
    #===========================
    if [ "$FW_PROTO" == "ipv4" ]; then true
    $IPTABLES -I INPUT -s 169.254.0.0/16 -m state --state RELATED,ESTABLISHED -j ACCEPT
    fi
    # IPv6 Custom Firewall Rules
    #===========================
    if [ "$FW_PROTO" == "ipv6" ]; then true
    fi
    [root@clearos ~]$


    Is that OK from your point of view? Thank you.

    Ondrej
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 05:23 PM - #Permalink
    Resolved
    0 votes
    That may work but is not the normal way of working. The Custom Firewall has two input boxes, one for IPv4 and one for IPv6. This populates the /etc/clearos/firewall.d/custom file which looks like:
    #######################################
    # Created by API - Please Do NOT Edit #
    #######################################

    # IPv4 Custom Firewall Rules
    #===========================

    if [ "$FW_PROTO" == "ipv4" ]; then true
    # Config file
    fi

    # IPv6 Custom Firewall Rules
    #===========================

    if [ "$FW_PROTO" == "ipv6" ]; then true
    fi
    when empty. The webconfig populates the relevant section. If you want to use the local file, can I just suggest you use the IPv4 section from this as a template? Again $FW_PROTO is set elsewhere and it is this which triggers $IPTABLES between using "iptables -w" and "ip6tables -w". If you use this template you can (should?) still use $IPTABLES.

    I did ask you to post the contents of your /etc/clearos/firewall.d/custom to see what you had done there. You would also have seen the layout.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 04:21 PM - #Permalink
    Resolved
    0 votes
    Hello Nick,

    I think I know now what you mean by the "ipv6 problem" above.


    [root@clearos firewall.d]# /usr/sbin/iptables -I INPUT -s 169.254.169.254 -j ACCEPT
    [root@clearos firewall.d]# /usr/sbin/ip6tables -I INPUT -s 169.254.169.254 -j ACCEPT
    ip6tables v1.4.21: host/network `169.254.169.254' not found
    Try `ip6tables -h' or 'ip6tables --help' for more information.


    In real, Centos Firewall Engine service is running, both ipv4 and ipv6 even if the service is marked as failed by Systemctl (because it runs the iptables rule for ip6tables). If IPv6 is not required, it is possible to comment the IPv6 command in exec-stop.sh and exec-start.sh file, but it is not "nice" solution. Does exists any way how to tell the firewall to use the specific rule either for IPv4 or for IPv6?

    I placed the following code into the local file which solves it, but it would be nice to have there e.g. [IPV4] [/IPV4] and [IPV6][/IPV6] section or something similar for the future.


    if [ "$IPTABLES" == "/sbin/iptables -w" ]; then
    $IPTABLES -I INPUT -s 169.254.0.0/24 -m state --state RELATED,ESTABLISHED -j ACCEPT
    elif [ "$IPTABLES" == "/sbin/ip6tables -w" ]; then
    #null
    fi


    I think it is finally solved. Thank you Nick for all your help.

    Regards
    Ondrej
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 01:50 PM - #Permalink
    Resolved
    0 votes
    $IPTABLES is defined in /usr/sbin/firewall-start which you can get to by examining the systemd unit file (/usr/lib/systemd/system/firewall.service) and tracing it through. If you don't want to use $IPTABLES, use "iptables -w" but don't use "iptables" on its own.

    What is the contents of your /etc/clearos/firewall.d/custom file?

    Perhaps a better, more target rule in line with the bug request I put is:
    $IPTABLES -I INPUT -s 169.254.0.0/24 -m state --state RELATED,ESTABLISHED -j ACCEPT
    What I have actually asked for is for the current incoming block on 169.254/16 just be limited to NEW connections, implicitly allowing in RELATED and ESTABLISHED which are explicitly allowed further down the rule chain.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, August 15 2018, 12:55 PM - #Permalink
    Resolved
    0 votes
    I don't know.
    The only error is the error I sent you. There is nothing more in /var/log/messages or /var/log/secure
    As I mentioned I converted to ClearOS from CentOS so maybe some definition of this variable is missing somewhere. I checked it

    Where is variable $IPTABLES defined? I found it in neither /etc/clearos/firewall.conf nor /etc/clearos/firewall.d/types. Does exist any other location?

    Thank you
    Ondrej
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, August 14 2018, 05:43 PM - #Permalink
    Resolved
    0 votes
    That error suggests you've applied the rule as an IpV6 rule and not an IPv4 rule?

    ... and I've filed the request - bug 21091.
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, August 14 2018, 05:29 PM - #Permalink
    Resolved
    0 votes
    It looks like it makes sense if I file an issue report to change the default rule just to block NEW incoming packets.
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, August 14 2018, 05:27 PM - #Permalink
    Resolved
    0 votes
    Hello Nick,

    When I use it on command line, it works, but I added the following line to /etc/clearos/firewall.d/local and the firewall is not able to start. Is that correct place where type the command:

    $IPTABLES -I INPUT -s 169.254.169.254 -j ACCEPT



    Output from journalctl -xe

    Aug 14 17:21:00 clearos firewall6[1925]: Running incoming denied rules
    Aug 14 17:21:00 clearos firewall6[1925]: Running user-defined incoming rules
    Aug 14 17:21:00 clearos firewall6[1925]: Running default incoming allowed rules
    Aug 14 17:21:00 clearos firewall6[1925]: Running Masquerading
    Aug 14 17:21:00 clearos firewall6[1925]: Enabling NAT on WAN interface eth0
    Aug 14 17:21:00 clearos firewall6[1925]: Running default forwarding rules
    Aug 14 17:21:00 clearos firewall6[1925]: Execution time: 0.103s
    Aug 14 17:21:00 clearos clearsyncd[678]: System Events: Socket hang-up: 26
    Aug 14 17:21:00 clearos firewall6[1972]: Running post-firewall:
    Aug 14 17:21:00 clearos firewall6[1973]: Running /etc/clearos/firewall.d/local
    Aug 14 17:21:00 clearos systemd[1]: firewall.service: main process exited, code=
    Aug 14 17:21:00 clearos systemd[1]: Failed to start ClearOS Firewall Engine.
    -- Subject: Unit firewall.service has failed
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit firewall.service has failed.
    --
    -- The result is failed.
    Aug 14 17:21:00 clearos systemd[1]: Unit firewall.service entered failed state.
    Aug 14 17:21:00 clearos systemd[1]: firewall.service failed.



    Thank you
    Ondrej
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, August 14 2018, 05:07 PM - #Permalink
    Resolved
    0 votes
    Hello Nick,

    Thank you for your quick answer. I will try what you recommended.
    Unfortunately, Google is not the only one who uses 169.254.169.254.
    https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

    https://docs.microsoft.com/en-us/azure/virtual-machines/windows/instance-metadata-service

    So three biggest providers use it. Would be possible to mark this as an enhancement request, please? It is small change which will affect only this case and maybe easy to implement.

    Thank you
    Ondrej
    The reply is currently minimized Show
Your Reply