Forums

t1ck3ts
t1ck3ts
Offline
Resolved
0 votes
So I've setup a 1-to-1 NAT for my second public IP Address to forward the connections to my internal address (192.168.1.1)
I've already told my ISP that its coming from the same MAC as the primary address is on.

Problem is, i cant make any connections from the outside (ie. FTP) but the kicker is, i can ping it from outside and it responds.

Setup a trace and it seems the ClearOS box is picking it up but not doing anything with it, could it be a possible iptable rule of some sort?

Apr 11 21:30:23 gateway kernel: TRACE: raw:PREROUTING:policy:2 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.xx LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10055 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402) 
Apr 11 21:30:23 gateway kernel: TRACE: mangle:PREROUTING:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10055 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:23 gateway kernel: TRACE: nat:PREROUTING:rule:31 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10055 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:23 gateway kernel: TRACE: mangle:INPUT:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10055 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:23 gateway kernel: TRACE: filter:INPUT:policy:36 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10055 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:26 gateway kernel: TRACE: raw:PREROUTING:policy:2 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10056 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:26 gateway kernel: TRACE: mangle:PREROUTING:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10056 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:26 gateway kernel: TRACE: nat:PREROUTING:rule:31 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10056 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:26 gateway kernel: TRACE: mangle:INPUT:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10056 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:26 gateway kernel: TRACE: filter:INPUT:policy:36 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x0A PREC=0x00 TTL=111 ID=10056 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:32 gateway kernel: TRACE: raw:PREROUTING:policy:2 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x08 PREC=0x00 TTL=111 ID=10057 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:32 gateway kernel: TRACE: mangle:PREROUTING:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x08 PREC=0x00 TTL=111 ID=10057 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:32 gateway kernel: TRACE: nat:PREROUTING:rule:31 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=xxx.xxx.xx.32 LEN=48 TOS=0x08 PREC=0x00 TTL=111 ID=10057 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:32 gateway kernel: TRACE: mangle:INPUT:policy:1 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x08 PREC=0x00 TTL=111 ID=10057 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B401010402)
Apr 11 21:30:32 gateway kernel: TRACE: filter:INPUT:policy:36 IN=enp2s0 OUT= MAC=d4:3d:7e:b8:0e:5a:00:25:90:e7:df:73:08:00 SRC=xx.xx.xxx.xx DST=192.168.1.1 LEN=48 TOS=0x08 PREC=0x00 TTL=111 ID=10057 DF PROTO=TCP SPT=62177 DPT=21 SEQ=1525807412 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B401010402)


Chain PREROUTING (policy ACCEPT 4913 packets, 307K bytes)
pkts bytes target prot opt in out source destination
19 1311 DNAT all -- * * 0.0.0.0/0 xxx.xxx.xx.32 to:192.168.1.1
0 0 ACCEPT tcp -- * * 0.0.0.0/0 192.168.1.1 tcp dpt:80
0 0 ACCEPT tcp -- * * 0.0.0.0/0 192.168.2.1 tcp dpt:80
3 156 ACCEPT tcp -- * * 0.0.0.0/0 xxx.xxx.xx.31 tcp dpt:80
42 2208 REDIRECT tcp -- enp4s1 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 redir ports 8080
0 0 REDIRECT tcp -- enp4s2 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 redir ports 8080

Chain INPUT (policy ACCEPT 1442 packets, 97796 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 399 packets, 31043 bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 80 packets, 5511 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * tun+ 0.0.0.0/0 0.0.0.0/0
14 2688 SNAT all -- * * 192.168.1.1 0.0.0.0/0 to:xxx.xxx.xx.32
0 0 SNAT all -- * * 192.168.1.0/24 192.168.1.1 to:192.168.1.1
0 0 SNAT all -- * * 192.168.2.0/24 192.168.1.1 to:192.168.2.1
1976 115K MASQUERADE all -- * enp2s0 0.0.0.0/0 0.0.0.0/0

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet xxx.xxx.xx.31 netmask 255.255.255.0 broadcast xxx.xxx.xx.255
inet6 fe80::d63d:7eff:feb8:e5a prefixlen 64 scopeid 0x20<link>
ether d4:3d:7e:b8:0e:5a txqueuelen 1000 (Ethernet)
RX packets 258520861 bytes 312407936425 (290.9 GiB)
--
enp2s0:200: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet xxx.xxx.xx.32 netmask 255.255.255.0 broadcast xxx.xxx.xx.255
ether d4:3d:7e:b8:0e:5a txqueuelen 1000 (Ethernet)

enp4s1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::2e0:4cff:fe22:140a prefixlen 64 scopeid 0x20<link>
ether 00:e0:4c:22:14:0a txqueuelen 1000 (Ethernet)
RX packets 269957521 bytes 63525000585 (59.1 GiB)
--
enp4s2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.2.1 netmask 255.255.255.0 broadcast 192.168.2.255
ether 00:e0:4c:22:14:0c txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0


02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device 7758
Kernel driver in use: r8169
Kernel modules: r8169
--
04:00.0 Ethernet controller: VIA Technologies, Inc. VT6105/VT6106S [Rhine-III] (rev 86)
Subsystem: D-Link System Inc DFE-530TX PCI Fast Ethernet Adapter (rev. C)
04:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller (rev 10)
Subsystem: Realtek Semiconductor Co., Ltd. RTL8169/8110 Family PCI Gigabit Ethernet NIC
Kernel driver in use: r8169
Kernel modules: r8169
04:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller (rev 10)
Subsystem: Realtek Semiconductor Co., Ltd. RTL8169/8110 Family PCI Gigabit Ethernet NIC
Kernel driver in use: r8169
Kernel modules: r8169


What DID work though, is adding a VLAN interface onto my WAN nic, the only problem is, my syswatch log file was getting flooded:

Tue Mar 27 03:57:35 2018  info: enp2s0.2 - waiting for static IP reconnect
Tue Mar 27 03:57:49 2018 info: enp2s0.2 - ping check on gateway failed - xxx.xxx.xx.1
Tue Mar 27 03:57:51 2018 debug: enp2s0.2 - ping check on server #1 failed - 8.8.8.8 (ping size: 1)
Tue Mar 27 03:57:53 2018 info: enp2s0.2 - ping check on server #1 failed - 8.8.8.8
Tue Mar 27 03:58:00 2018 info: enp2s0.2 - ping check on server #2 failed - 54.152.208.245
Tue Mar 27 03:58:00 2018 warn: enp2s0.2 - connection is down


If the 1-to-1 NAT is not an option, is there a way to stop the flooding of syswatch log?
Wednesday, April 11 2018, 07:58 PM
Share this post:
Responses (5)
  • Accepted Answer

    Wednesday, April 11 2018, 08:53 PM - #Permalink
    Resolved
    0 votes
    I like the trace. I've never seen it before.

    You appear to have a secondary issue in that you have both an RTL8168 and RTL8169 NIC, and the built-in r8169 driver is not good with the RTL8168. Assuming you are on 7.x, please can you do a:
    yum install kmod-r816*
    Then reboot. The lspci should then show the the r8168 driver in use for the RTL8111/8168/8411 card.

    If you want it I also have a driver for the Via Rhine card here.

    Now to your main problem. Remember the incoming firewall is for traffic destined for ClearOS and port forwarding is for traffic through ClearOS to the LAN behind. 1-to-1 is effectively port forwarding in conjunction with setting up a virtual IP address on the WAN port.

    I am not sure if it will work so first of all make a copy of your /etc/sysconfig/network-scripts/ifcfg-enp2s0:200 and put it somewhere else. The remove the 1-to-1 NAT and just try an incoming firewall rule at the command line:
    iptables -I INPUT -d xxx.xxx.xx.32 -p tcp --dport 21 -j ACCEPT
    If it works, change "iptables" to "$IPTABLES" and use it as a custom firewall rule.

    If this does not work also add an irrelevant 1-to-1 NAT rule to an oddball port and a non-existent IP address, then you have your virtual IP and your incoming firewall rule should work. If it does work, you should be able to remove the 1-to-1 NAT again and put back the /etc/sysconfig/network-scripts/ifcfg-enp2s0:200 file you took out. Then I'm not sure if the next stage. Restarting networking is brutal but should work. You may be able to down and up your WAN interface and that may sort it. I'd have to check tomorrow, so if you get it working today, let me know. There is a chance you need to modify /etc/clearos/network.conf - you'll be able to tell if setting up the 1-to-1 NAT has put an entry for the virtual interface in it.

    [edit]
    There is some rubbish there. Just try setting up a virtual interface on your WAN port and an incoming firewall rule.
    [/edit]
    The reply is currently minimized Show
  • Accepted Answer

    t1ck3ts
    t1ck3ts
    Offline
    Wednesday, April 11 2018, 09:20 PM - #Permalink
    Resolved
    0 votes
    Well, the Virtual IP address and manual iptable rule did the trick! Thanks Nick.

    The only problem now is, the FTP is not sending me to the flexshare ftp folder it should be sending me to.
    I get to the flexshare folder if i FTP to my Public Address #1, just doesn't work for Public Address #2

    Any ideas whats up with that?

    edit:

    The only thing i can see is the following, that could be causing it:

    Apr 11 23:21:00 gateway proftpd[17130]: 127.0.0.1 (xx.xx.xxx.46[xx.xx.xxx.46]) - FTP session opened.
    Apr 11 23:21:01 gateway systemd-logind: New session 9241 of user remoteusertest.
    Apr 11 23:21:12 gateway proftpd[17131]: xxx.xxx.xx.31 (xx.xx.xxx.46[xx.xx.xxx.46]) - FTP session opened.
    Apr 11 23:21:12 gateway systemd-logind: New session 9242 of user remoteusertest.
    Apr 11 23:21:38 gateway proftpd[17131]: xxx.xxx.xx.31 (xx.xx.xxx.46[xx.xx.xxx.46]) - FTP session closed.
    Apr 11 23:21:38 gateway systemd-logind: Removed session 9242.
    Apr 11 23:21:58 gateway proftpd[17130]: 127.0.0.1 (xx.xx.xxx.46[xx.xx.xxx.46]) - FTP session closed.
    Apr 11 23:21:58 gateway systemd-logind: Removed session 9241.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 11 2018, 09:30 PM - #Permalink
    Resolved
    0 votes
    FTP configs are in /etc/proftpd.d/flex-21.conf and that is auto-generated. Any changes you make like adding IP2 risk being lost nest time you visit the Flexshare FTP settings unless you set the immutable but (chattr +i /etc/proftpd.d/flex-21.conf), but then you need to remember you've set it. You could try going into the flexshare settings and making a trivial edit and see if the file regenerates correctly. If not, it is a manual edit.
    The reply is currently minimized Show
  • Accepted Answer

    t1ck3ts
    t1ck3ts
    Offline
    Wednesday, April 11 2018, 09:31 PM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    FTP configs are in /etc/proftpd.d/flex-21.conf and that is auto-generated. Any changes you make like adding IP2 risk being lost nest time you visit the Flexshare FTP settings unless you set the immutable but (chattr +i /etc/proftpd.d/flex-21.conf), but then you need to remember you've set it. You could try going into the flexshare settings and making a trivial edit and see if the file regenerates correctly. If not, it is a manual edit.


    Yeah, that's exactly what i was playing with!

    Had to add public ip addres #2 to the <VirtualHost> tag.

    Hmmm, dont suppose you know where it gets the IP addresses when generating the config, do you? Maybe a edit of that could add the Virtual IP NIC
    The reply is currently minimized Show
  • Accepted Answer

    Thursday, April 12 2018, 10:39 AM - #Permalink
    Resolved
    0 votes
    It is difficult when you can't really read php, but let's try.

    The flex-21 file seems to be controlled by /usr/clearos/apps/ftp/libraries/ProFTPd.php around line 2735 but that calls Iface_Manager which is /usr/clearos/apps/network/libraries/Iface_Manager.php. Interfaces seem to be picked up from around line 162 but that is a comment block. Line 210 appears to ignore interfaces with a : in the name but commenting it out does not help. There is more at line 452, but flipping that to FALSE also does not help. At one point the file /etc/clearos/network.conf is read and that does not contain a reference to the virtual interface so I don't know if that is a problem in that the php then never tries to enumerate the virtual interface.

    Good luck!
    The reply is currently minimized Show
Your Reply