Community Forum

Sandeep
Sandeep
Offline
Resolved
0 votes
Hi,

I am facing a strange problem i have 4 Machines behind the clearos Firewall, out of 4, three are able to ping and able to access repository server which placed at tokyo (Cloud provider mapped it locally, hence it is accessible with local nic), but one machine is not able to communicate with it. my setup looks like below

---->Clearos ------>eno1 -----> all machines
------>eno2

eno1 = local
eno2 = Public

As the setup is placed in cloud, all the traffic by default passing from eno1 through their internal routing, but public access and communication not possible without public ip.
There is no firewall enabled in internal machine, hence routing is same for all machines (gateway is firewall eno1 nic)
After consulting with the cloud provider they ask to allow some subnets as per their pre-requisites which i have allowed by putting in custom-firewall rule as below:

$IPTABLES -I FORWARD -s PublicCloudSubnet -d LocalLansubnet -j ACCEPT
$IPTABLES -I FORWARD -d PublicCloudSubnet -s LocalLansubnet -j ACCEPT

But still it is not working, if i remove that machine from the firewall it start accessing the repository server

Could you please help?
Monday, December 04 2017, 07:19 AM
Share this post:
Responses (8)
  • Accepted Answer

    Wednesday, December 06 2017, 08:38 PM - #Permalink
    Resolved
    0 votes
    I understand that you may want the port forwarding rules in place, but all I am asking is that you do a quick test by removing the rules and seeing if you can then access your repo. Add the port forwarding rules straight after your test.

    The rules you've proposed will do nothing on their own. The first one should never do anything as no traffic should ever appear on your WAN carrying a LAN destination IP. It will only carry your WAN IP or it won't know how to find you. The second rule will always work and is unnecessary as the traffic is allowed by default.

    I am struggling a bit with your set up because it defies normality as it involves routing private IP addresses over the internet which does not normally work. "ping 10.3.65.129" should never work via the internet unless it is within your ISP and, even then, it is questionable.

    Can you give a network diagram between your LAN and your repo LAN detailing the IP's and subnets?
    The reply is currently minimized Show
  • Accepted Answer

    Sandeep
    Sandeep
    Offline
    Wednesday, December 06 2017, 03:25 AM - #Permalink
    Resolved
    0 votes
    Actually I need that port-forwarding rules, so i can access the LAN machine because my infra is at Cloud and their is no way to access the machine without the firewall public ip
    what i suspecting --- the repo is only accessible via local LAN of firewall.
    By default LAN machines forwarding traffic from local Nic to Public Nic, for particular server if we can move traffic of repo access from Local Nic to Local Nic then I believe it solves the problem, but I don't know how to do that.
    If you see the last output, from app002 machine it sending packets to Firewall Local nic and then traffic moves to public gateway. But for nginxmul this is not happening as it only sending packets to firewall and then it didn't find the gateway to move the traffic.

    I have a discussion with cloud provider they have given some series of IP subnets, and asked to open up in firewall. Could you please confirm if i will apply the below rule will it open the traffic or I need to do something else
    $IPTABLES -I FORWARD -s PublicCloudSubnet -d LocalLansubnet -j ACCEPT
    $IPTABLES -I FORWARD -d PublicCloudSubnet -s LocalLansubnet -j ACCEPT
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, December 05 2017, 03:44 PM - #Permalink
    Resolved
    0 votes
    It is a strange set up that you have. You should not be able to route to a 10.x.y.z address via a public IP unless the private IP exists within the public IP's LAN and even then you should not really be able to.

    When you do port forwarding, there are a couple of PREROUTING and POSTROUTING rules which are added as well as a FORWARD rule. I am wondering if one of these is interfering. Can you try to remove the port forwarding for the moment?
    The reply is currently minimized Show
  • Accepted Answer

    Sandeep
    Sandeep
    Offline
    Tuesday, December 05 2017, 12:54 PM - #Permalink
    Resolved
    0 votes
    I don't think it is effect, as I mapped it from firewall itself, also my repo is only accessible from local NIC = eno1 of firewall having ip 10.162.34.249 and their is no Ipsec tunnel required for that

    below are the trace-path info from firewall, the machine having issue and other machine in the same LAN

    Output from Firewall
    -----------------------
    [root@firewall ~]# ping
    PING 10.3.65.129 (10.3.65.129) 56(84) bytes of data.
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=1 ttl=55 time=112 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=2 ttl=55 time=112 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=3 ttl=55 time=112 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=4 ttl=55 time=112 ms
    ^C
    --- 10.3.65.129 ping statistics ---
    4 packets transmitted, 4 received, 0% packet loss, time 3003ms
    rtt min/avg/max/mdev = 112.107/112.110/112.119/0.334 ms
    [root@firewall ~]# tracepath -n 10.3.65.129
    1?: [LOCALHOST] pmtu 1500
    1: no reply
    2: no reply
    3: no reply
    4: no reply
    5: no reply
    6: no reply
    7: no reply
    8: no reply
    9: no reply
    10: 10.3.65.129 112.419ms reached
    Resume: pmtu 1500 hops 10 back 10

    root@firewall ~]# route -n
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 169.38.65.161 0.0.0.0 UG 0 0 0 eno2
    10.0.0.0 10.162.34.193 255.0.0.0 UG 0 0 0 eno1
    10.162.34.192 0.0.0.0 255.255.255.192 U 0 0 0 eno1
    161.26.0.0 10.162.34.193 255.255.0.0 UG 0 0 0 eno1
    169.38.65.0 0.0.0.0 255.255.255.0 U 0 0 0 eno2
    169.38.74.0 0.0.0.0 255.255.255.0 U 0 0 0 eno2
    [root@firewall ~]#

    [root@ngnixmule ~]# tracepath -n 10.3.65.129
    1?: [LOCALHOST] pmtu 1500
    1: 10.162.34.249 0.877ms
    1: 10.162.34.249 0.881ms
    2: no reply
    3: no reply
    4: no reply
    5: no reply
    6: no reply
    7: no reply
    8: no reply
    9: no reply
    10: no reply
    11: no reply
    12: no reply
    13: no reply
    14: no reply
    15: no reply
    16: no reply
    17: no reply
    18: no reply
    19: no reply
    20: no reply
    21: no reply
    22: no reply
    23: no reply
    24: no reply
    25: no reply
    26: no reply
    27: no reply
    28: no reply
    29: no reply
    30: no reply
    31: no reply
    Too many hops: pmtu 1500
    Resume: pmtu 1500
    [root@ngnixmule ~]#
    [root@ngnixmule ~]# route -n
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 10.162.34.249 0.0.0.0 UG 0 0 0 eth0
    10.162.34.192 0.0.0.0 255.255.255.192 U 0 0 0 eth0
    169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
    You have new mail in /var/spool/mail/root

    [root@app002 ~]# tracepath -n rhncaptok0202.service.networklayer.com
    1?: [LOCALHOST] pmtu 1500
    1: 10.162.34.249 0.817ms
    1: 10.162.34.249 0.813ms
    2: 169.254.157.91 1.654ms asymm 1
    3: 169.254.61.126 1.478ms asymm 2
    4: 169.254.2.144 1.561ms asymm 3
    5: no reply
    6: no reply
    7: no reply
    8: no reply
    9: no reply
    10: no reply
    11: 10.3.65.129 99.414ms reached
    Resume: pmtu 1500 hops 11 back 10

    [root@app002 ~]# route -n
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 10.162.34.249 0.0.0.0 UG 0 0 0 eth0
    10.162.34.192 0.0.0.0 255.255.255.192 U 0 0 0 eth0
    169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0

    [root@kiteapp002 ~]# ping 10.3.65.129
    PING 10.3.65.129 (10.3.65.129) 56(84) bytes of data.
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=1 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=2 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=3 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=4 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=5 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=6 ttl=55 time=101 ms
    64 bytes from 10.3.65.129 (10.3.65.129): icmp_seq=7 ttl=55 time=101 ms
    ^C
    --- 10.3.65.129 ping statistics ---
    7 packets transmitted, 7 received, 0% packet loss, time 6009ms
    rtt min/avg/max/mdev = 101.565/101.638/101.714/0.420 ms

    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, December 05 2017, 12:25 PM - #Permalink
    Resolved
    0 votes
    A quick idea. You are also port forwarding 80 and 443 to 10.162.34.212. If you remove the port forwards does your repo then work?

    Your repo is using a private IP address. Is it at the other end of your IPsec VPN tunnel? If not, what is the route to it?
    The reply is currently minimized Show
  • Accepted Answer

    Sandeep
    Sandeep
    Offline
    Tuesday, December 05 2017, 08:01 AM - #Permalink
    Resolved
    0 votes
    Hi Nick,

    Here is the output

    10.162.34.212 is my machine which not able to communicate to my reposerver = 10.3.65.129

    ====================

    iptables -nlv
    ===============
    Chain INPUT (policy DROP 44 packets, 4676 bytes)
    pkts bytes target prot opt in out source destination
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_INGRESS src
    16 1216 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:123 state RELATED,ESTABLISHED
    51 2934 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
    0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x12/0x12 state NEW reject-with tcp-reset
    6 2609 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x17/0x02 state NEW
    0 0 DROP all -- eno2 * 127.0.0.0/8 0.0.0.0/0
    0 0 DROP all -- eno2 * 169.254.0.0/16 0.0.0.0/0
    76 8456 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
    92 9679 ACCEPT all -- eno1 * 0.0.0.0/0 0.0.0.0/0
    7 203 ACCEPT icmp -- eno2 * 0.0.0.0/0 0.0.0.0/0 icmptype 0
    0 0 ACCEPT icmp -- eno2 * 0.0.0.0/0 0.0.0.0/0 icmptype 3
    2 192 ACCEPT icmp -- eno2 * 0.0.0.0/0 0.0.0.0/0 icmptype 8
    0 0 ACCEPT icmp -- eno2 * 0.0.0.0/0 0.0.0.0/0 icmptype 11
    0 0 ACCEPT udp -- eno2 * 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
    0 0 ACCEPT tcp -- eno2 * 0.0.0.0/0 0.0.0.0/0 tcp spt:67 dpt:68
    319 19472 ACCEPT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:3394
    534 132K ACCEPT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:81
    65 7280 ACCEPT udp -- * * 0.0.0.0/0 169.38.65.167 udp spt:500 dpt:500
    736 318K ACCEPT esp -- * * 0.0.0.0/0 169.38.65.167
    0 0 ACCEPT ah -- * * 0.0.0.0/0 169.38.65.167
    0 0 ACCEPT all -- * * 0.0.0.0/0 169.38.65.167 mark match 0x64
    0 0 ACCEPT all -- * * 0.0.0.0/0 10.162.34.249 mark match 0x64
    83 15660 ACCEPT udp -- eno2 * 0.0.0.0/0 0.0.0.0/0 udp dpts:1024:65535 state RELATED,ESTABLISHED
    0 0 ACCEPT tcp -- eno2 * 0.0.0.0/0 0.0.0.0/0 tcp dpts:1024:65535 state RELATED,ESTABLISHED

    Chain FORWARD (policy DROP 29 packets, 1276 bytes)
    pkts bytes target prot opt in out source destination
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_SELF src,dst,dst
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_EGRESS dst
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_INGRESS src
    60 9048 ACCEPT all -- * * 10.118.1.0/24 10.162.34.192/26
    0 0 ACCEPT all -- * * 10.118.1.0/24 10.162.34.192/26
    0 0 ACCEPT all -- * * 161.202.118.0/23 10.162.34.192/26
    730 275K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x64
    0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 10.162.34.194 tcp dpt:3389
    0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 10.162.34.212 tcp dpt:22
    0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 10.162.34.202 tcp dpt:22
    0 0 ACCEPT tcp -- * eno1 0.0.0.0/0 10.162.34.245 tcp dpt:22
    1587 574K ACCEPT tcp -- * eno1 0.0.0.0/0 10.162.34.242 tcp dpt:22
    0 0 ACCEPT icmp -- eno2 * 0.0.0.0/0 10.162.34.212 icmptype 0
    52 5772 ACCEPT icmp -- eno2 * 0.0.0.0/0 10.162.34.212 icmptype 3
    0 0 ACCEPT icmp -- eno2 * 0.0.0.0/0 10.162.34.212 icmptype 8
    1 111 ACCEPT icmp -- eno2 * 0.0.0.0/0 10.162.34.212 icmptype 11
    0 0 DROP icmp -- eno2 * 0.0.0.0/0 10.162.34.212
    0 0 ACCEPT tcp -- eno2 * 0.0.0.0/0 10.162.34.212 tcp dpt:80
    3035 492K ACCEPT tcp -- eno2 * 0.0.0.0/0 10.162.34.212 tcp dpt:443
    9007 2586K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
    0 0 ACCEPT all -- pptp+ * 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- tun+ * 0.0.0.0/0 0.0.0.0/0
    21566 61M ACCEPT all -- eno1 * 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- * * 172.18.31.50 10.100.106.73
    0 0 ACCEPT all -- * * 10.100.106.73 172.18.31.50


    iptables -nlv -t nat
    ======================
    Chain OUTPUT (policy DROP 0 packets, 0 bytes)
    pkts bytes target prot opt in out source destination
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_SELF src,dst,dst
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set snortsam_EGRESS dst
    76 8456 ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- * pptp+ 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
    91 9219 ACCEPT all -- * eno1 0.0.0.0/0 0.0.0.0/0
    9 395 ACCEPT icmp -- * eno2 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT udp -- * eno2 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
    0 0 ACCEPT tcp -- * eno2 0.0.0.0/0 0.0.0.0/0 tcp spt:68 dpt:67
    209 19892 ACCEPT tcp -- * eno2 169.38.65.167 0.0.0.0/0 tcp spt:3394
    483 237K ACCEPT tcp -- * eno2 169.38.65.167 0.0.0.0/0 tcp spt:81
    65 7280 ACCEPT udp -- * eno2 169.38.65.167 0.0.0.0/0 udp spt:500 dpt:500
    823 166K ACCEPT esp -- * eno2 169.38.65.167 0.0.0.0/0
    0 0 ACCEPT ah -- * eno2 169.38.65.167 0.0.0.0/0
    72 32612 ACCEPT all -- * eno2 0.0.0.0/0 0.0.0.0/0

    Chain DROP-lan (0 references)
    pkts bytes target prot opt in out source destination
    0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0


    Chain PREROUTING (policy ACCEPT 451 packets, 34082 bytes)
    pkts bytes target prot opt in out source destination
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:9005 to:10.162.34.194:3389
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:9001 to:10.162.34.212:22
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:9002 to:10.162.34.202:22
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:9003 to:10.162.34.245:22
    1 52 DNAT tcp -- * * 0.0.0.0/0 169.38.65.167 tcp dpt:9004 to:10.162.34.242:22
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.38.74.92 tcp dpt:80 to:10.162.34.212
    206 12953 DNAT tcp -- * * 0.0.0.0/0 169.38.74.92 tcp dpt:443 to:10.162.34.212

    Chain INPUT (policy ACCEPT 106 packets, 8597 bytes)
    pkts bytes target prot opt in out source destination

    Chain OUTPUT (policy ACCEPT 54 packets, 3271 bytes)
    pkts bytes target prot opt in out source destination

    Chain POSTROUTING (policy ACCEPT 257 packets, 16156 bytes)
    pkts bytes target prot opt in out source destination
    44 2640 SNAT all -- * * 0.0.0.0/0 10.100.106.73 to:172.18.31.50
    0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
    0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 policy match dir out pol ipsec
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.194 tcp dpt:3389 to:10.162.34.249
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.212 tcp dpt:22 to:10.162.34.249
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.202 tcp dpt:22 to:10.162.34.249
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.245 tcp dpt:22 to:10.162.34.249
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.242 tcp dpt:22 to:10.162.34.249
    16 960 SNAT all -- * * 10.162.34.212 0.0.0.0/0 to:169.38.74.92
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.212 tcp dpt:80 to:10.162.34.249
    0 0 SNAT tcp -- * * 10.162.34.192/26 10.162.34.212 tcp dpt:443 to:10.162.34.249
    163 9844 MASQUERADE all -- * eno2 0.0.0.0/0 0.0.0.0/0


    Thanks
    Sandeep
    The reply is currently minimized Show
  • Accepted Answer

    Sandeep
    Sandeep
    Offline
    Monday, December 04 2017, 12:56 PM - #Permalink
    Resolved
    0 votes
    Hi Nick,

    Thanks for your reply, i have replied you in private, could you please share your comments after looking it
    The reply is currently minimized Show
  • Accepted Answer

    Monday, December 04 2017, 12:20 PM - #Permalink
    Resolved
    0 votes
    Unless you have changed the default policy of the Egress firewall to "block all, specify allowed destinations" then those rules should not be needed. the default is to allow all traffic out and all return traffic back in.

    What is the result of:
    iptables -nvL
    iptables -nvL -t nat
    And please put the results between "code tags" (the piece of paper icon with <> on it)

    What is the LAN IP of the machine being blocked?
    The reply is currently minimized Show
Your Reply