Hi all,

Since this thread kept coming up every time I googled for a fix for my pppoe slow speed issue, I thought I would share the steps I took to fix it on mine.

I had the similar symptoms as the OP - pppoe works fine using a generic router/modem type thing at near the full speed of my ISP connection (Fibre to the premise, theoretical link at 100mbps, I could generally get 95mbps or so from speedtest.net), but under ClearOS (running on a celeron intel NUC with a usb gigabit ethernet dongle as the second NIC) it was only getting around 50mbps.

Using the hints in here and in the linked bug report - https://tracker.clearos.com/view.php?id=20251 - I managed to get back to ~95mbps. I am not exactly sure which commands were needed, or which will survive modifying anything in the webgui or a reboot, but.... for now, it's working so I am happy.

In the webgui, open the marketplace and install "custom firewall".
Install, update dashboard menu, then go navigate to it in the network-> firewall section

Under IPv4 rules click "add" and paste in the following:
iptables -t mangle -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

add a comment like "fix pppoe maybe" and save the rule.

I know this alone wasn't enough, so I went deeper....

ssh into the clearOS device as root.

run the following commands.

*** backup your config, just in case (yes, I did need it because I ran the command after this with > instead of >> the first time )
# cp /etc/sysconfig/network-scripts/ifcfg-ppp0 ifcfg-ppp0.bkp

*** add the kernel module line to the config
# echo "LINUX_PLUGIN=/usr/lib64/pppd/2.4.5/rp-pppoe.so" >> /etc/sysconfig/network-scripts/ifcfg-ppp0

*** this thing to make it an option?
# echo "plugin rp-pppoe.so" >> /etc/ppp/options

*** reset the connection - replace ppp0 with your ppp interface ID
# ifdown ppp0
# ifup ppp0

after doing all that, speedtest was finally running fast again.

I did also muck about a lot with MTU size on various interfaces but none of that made a difference, so I don't think it's relevant....probably.

Hope that helps the next person to come along.

I see there's no response to that previous post from Ryan Anthony, but it deserves one.

I've been tearing my hair out over a PPP via ADSL link which would only run at about 1.5 mbps down and 0.2 mbps up. With a standard cheap modem it was 18 and 1 mbps, about 12 times faster. I tried Ryan's suggestion to disable the Bandwidth & QoS manager, and speeds immediately returned to 18/1. Wheee!

I noted in the documentation that having too small an upload speed configured in the Bandwidth & QoS manager could adversely affect speeds, so I tried it again but configured the speeds as 30000 (30 mbps) down and 2000 up. With Bandwidth QoS manager running and configured with these speeds it ran at the same high speed.

Then I made the mistake of hitting the "Save speed test results" button on the speed test results dialog. This mashed the gains and we were back at 1.5 / 0.2.

Moral of the story - either don't use the Bandwidth & QoS manager, or set the speeds good and high (like maybe double what you expect) and NEVER HIT the "Save speed test results" button after running a speed test.

A bug fix would be nice. Even just removing the button would be an improvement.

I had to institute MSS Clamping in order for the websites to show up. So if anyone has the magical combo of ClearOS + CenturyLink + speed and website issues, you may want to try:

edit: /etc/clearos/firewall.d/custom

then add the following line after "if [ "$FW_PROTO" == "ipv4" ]; then true"

$IPTABLES -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Thanks for the feedback.

There is only reference to the bug in this thread, but if you look at it, it recommends the following rule is added:

$IPTABLES -t mangle -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Most references put this in the mangle table rather than the filter table (the default if not specified).

Can I also suggest you use the Custom Firewall module to add these rules? They also add a comment to the file.

Hello Chris,

Thanks for your post.

The patch for this as a permanent configuration is trivially easy. In /usr/clearos/apps/network/libraries/Iface.php, somewhere between line 1646 and 1668, add a line:

        $info['LINUX_PLUGIN'] = '/usr/lib64/pppd/2.4.5/rp-pppoe.so';

My section looks like:

        $info = array();

        $info['DEVICE'] = $eth;

        $info['TYPE'] = self::TYPE_PPPOE;

        $info['USERCTL'] = 'no';

        $info['BOOTPROTO'] = 'dialup';

        $info['NAME'] = 'DSL' . $eth;

        $info['ONBOOT'] = 'yes';

        $info['PIDFILE'] = '/var/run/pppoe-' . $eth . '.pid';

        $info['FIREWALL'] = 'NONE';

        $info['PING'] = '.';

        $info['PPPOE_TIMEOUT'] = '80';

        $info['LCP_FAILURE'] = '5';

        $info['LCP_INTERVAL'] = '20';

        $info['CLAMPMSS'] = '1412';

        $info['CONNECT_POLL'] = '6';

        $info['CONNECT_TIMEOUT'] = '80';

        $info['DEFROUTE'] = 'yes';

        $info['SYNCHRONOUS'] = 'no';

        $info['ETH'] = $physdev;

        $info['PROVIDER'] = 'DSL' . $eth;

        $info['PEERDNS'] = ($peerdns) ? 'yes' : 'no';

        $info['USER'] = $username;

        $info['LINUX_PLUGIN'] = '/usr/lib64/pppd/2.4.5/rp-pppoe.so';



        if (!empty($mtu))

            $info['MTU'] = $mtu;

You also need the firewall rule, but that should also be a one line patch if you use a sledgehammer and apply it all the time (which is safe as it only targets a ppp interface). If you need to detect a configured ppp interface it becomes more complicated.

The problem is this is a big change in PPPoE configuration and hence there is a lot of nervousness about breaking customer systems, especially as only one of the devs had a PPPoE line at the time of the thread. He has moved on, but in Feb I switched to PPPoE so I have been using it since then, but my line is only 40mbps down. If we could get more of the user base to trial it, we could get some confidence and, perhaps, switch. Perhaps we should try pushing it to the Community for a while, but there is a risk of a lot of disgruntled users if it does not work as expected.

Note that with kernel mode PPPoE, the MTU setting does nothing, which is why the firewall rule is needed. One possible issue is https sites breaking without the correct MTU, but I have not been seeing this so I am fairly confident.

Also worth noting is the synchronous and clampmss lines do nothing and other lines may become redundant as well.

No, it is not a known bug. I've seen complaints where the speed drops to 600Mbps which is way faster than 100Mbps.

Just to do a quick driver check, can you please give the output to:

lspci -k | grep Eth -A 3

Also you you notice any lights on your NIC or modem indicating the NIC may have dropped out of gigabit mode and reverted to 10/100 mode? Often a different combination or colour of lights will show.

Here's my lspci output.

[root@gateway ~]# lspci -k | grep Eth -A 3
01:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
Subsystem: Intel Corporation Device 0000
Kernel driver in use: e1000e
Kernel modules: e1000e
02:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
Subsystem: Intel Corporation Device 0000
Kernel driver in use: e1000e
Kernel modules: e1000e
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
Subsystem: Intel Corporation Device 0000
Kernel driver in use: e1000e
Kernel modules: e1000e
04:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
Subsystem: Intel Corporation Device 0000
Kernel driver in use: e1000e
Kernel modules: e1000e

I haven't noticed any change indicating a drop out of gigabit. When I check it with ifconfig, it shows gigabit.

Your NIC drivers are OK.

I am not aware that ifconfig can give link speeds. "ethtool your_interface_name" will show it as the "Speed" value

Sorry, had ifconfig on the brain. Ethtool shows:

Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: off (auto)
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link

Nick Howitt wrote:

Your NIC drivers are OK.

I am not aware that ifconfig can give link speeds. "ethtool your_interface_name" will show it as the "Speed" value

As a thought, there has been an issue with the e1000e driver, but it tended to crash it rather than slow it. Can you try:

ethtool -K your_interface tso off

For the interface, use the underlying interface name and not ppp0. You can try it for all four since they use the same driver. It won't survive a reboot but that can be fixed if it works.

That made no difference. Below are screen shots of the speed tests.

Nick Howitt wrote:

As a thought, there has been an issue with the e1000e driver, but it tended to crash it rather than slow it. Can you try:

ethtool -K your_interface tso off

For the interface, use the underlying interface name and not ppp0. You can try it for all four since they use the same driver. It won't survive a reboot but that can be fixed if it works.

Is there any update on this? I won't be renewing my clearOS subscription if this can't be fixed. Without this, ClearOS doesn't have a lot of value to me.

Bane wrote:

That made no difference. Below are screen shots of the speed tests.

Nick Howitt wrote:

As a thought, there has been an issue with the e1000e driver, but it tended to crash it rather than slow it. Can you try:

ethtool -K your_interface tso off

For the interface, use the underlying interface name and not ppp0. You can try it for all four since they use the same driver. It won't survive a reboot but that can be fixed if it works.

Sorry I can't do any more. I don't have a DSL connection or your hardware to test with - and 1Gb connections don't exist here! I don't know what version of the e1000e driver will come with the ClearOS 7.5 kernel which is due soon.

Do you have another NIC you can test with?

All the interfaces on the unit are the same intel card. It is a four port NIC. So, do you mean another port or a completely different card.

Since the unit is a small form factor unit with four ports built in, I would have to use a USB Ethernet adapter to change NIC manufacturers, etc.

The unit itself is a quad core intel j1900D, I have never seen the unit use more than one code with clear OS. (https://www.amazon.com/dp/B072ZTCNLK/ref=sspa_dk_detail_5?psc=1&pd_rd_i=B072ZTCNLK&pd_rd_wg=9FgCI&pd_rd_r=453SYX5DAPFXG1P9P341&pd_rd_w=uVtqR)

Nick Howitt wrote:

Do you have another NIC you can test with?

You could try a different port first off, but I did mean a completely different NIC using different drivers.

PPPoE has an additional overhead of 8 bytes per packet. This will reduce the speed slightly but not enough to account for the difference you are seeing of itself i.e. a loss of about 0.53%

Incidentally is is interesting from your tests that the upload speed was somewhat faster while the upload was way down. The test results would be more valid if both tests used the same destination within a few minutes of each other. I know that Wyoming is adjacent to Colorado, but that does not necessarily mean the internet paths are equivalent and comparable.

However - those 8 extras bytes may be causing fragmentation and thus packet re-transmission. What happens if you try with a MTU of 1492 or slightly lower,assuming here, of course, it is by default 1500? Are you using synchronous or asynchronous mode for PPPoE? Synchronous mode requires a fast CPU - nut sure a J1900 would qualify...

Lastly, your link shows a system with an Atom E3845 which is not a J1900... though suspect your unit looks similar - just a newer model?

Please can you let me know if this issue is the same as ticket 558584 at Clearcenter?

Here's my CPU info. I it has four codes, but I have left out the other 3 sets of info, for brevity sake. My clearos box has 8gb of RAM. I am using the default PPPoE settings in clearOS. ow can I tell if it is sync or async mode?

[root@gateway ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 55
model name : Intel(R) Celeron(R) CPU J1900 @ 1.99GHz
stepping : 8
microcode : 0x829
cpu MHz : 1332.826
cache size : 1024 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch epb tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms dtherm ida arat
bogomips : 3993.60
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

Nick Howitt wrote:

Please can you let me know if this issue is the same as ticket 558584 at Clearcenter?

Tony Ellis wrote:

PPPoE has an additional overhead of 8 bytes per packet. This will reduce the speed slightly but not enough to account for the difference you are seeing of itself i.e. a loss of about 0.53%

Incidentally is is interesting from your tests that the upload speed was somewhat faster while the upload was way down. The test results would be more valid if both tests used the same destination within a few minutes of each other. I know that Wyoming is adjacent to Colorado, but that does not necessarily mean the internet paths are equivalent and comparable.

However - those 8 extras bytes may be causing fragmentation and thus packet re-transmission. What happens if you try with a MTU of 1492 or slightly lower,assuming here, of course, it is by default 1500? Are you using synchronous or asynchronous mode for PPPoE? Synchronous mode requires a fast CPU - nut sure a J1900 would qualify...

Lastly, your link shows a system with an Atom E3845 which is not a J1900... though suspect your unit looks similar - just a newer model?

Nick Howitt wrote:

Please can you let me know if this issue is the same as ticket 558584 at Clearcenter?

I can't see that ticket, so I am not sure if it is the same issue or not.

I have tried multiple cables and changed interfaces. The second I turn off PPPoE my bandwidth goes up to 800 to 900 mbps. I am seeing the degradation no matter what interface I use for PPPoE. I am not seeing high CPU or high memory usage when enabling PPPoE.

Nick Howitt wrote:

You could try a different port first off, but I did mean a completely different NIC using different drivers.

Can you give me the output of:

ifconfig

Also have you tried the latest kmod-e1000e driver available from here. You will need to reboot afterwards.

Below is the ifconfig output. I do not have PPPoE turned on right now. I can send another after I update the drivers and try PPPoE again.

enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.2.234 netmask 255.255.255.0 broadcast 192.168.2.255
inet6 fe80::2ba:abff:fe10:2460 prefixlen 64 scopeid 0x20<link>
ether 00:ba:ab:10:24:60 txqueuelen 1000 (Ethernet)
RX packets 1315135 bytes 1620145302 (1.5 GiB)
RX errors 0 dropped 1293 overruns 0 frame 0
TX packets 1220650 bytes 1377451410 (1.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0900000-d0920000

enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.3.7 netmask 255.255.255.0 broadcast 192.168.3.255
inet6 fe80::2ba:abff:fe10:2463 prefixlen 64 scopeid 0x20<link>
ether 00:ba:ab:10:24:63 txqueuelen 1000 (Ethernet)
RX packets 1243405 bytes 1382261019 (1.2 GiB)
RX errors 0 dropped 390 overruns 0 frame 0
TX packets 1312026 bytes 1620684828 (1.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 19 memory 0xd0600000-d0620000

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 13365 bytes 1472037 (1.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13365 bytes 1472037 (1.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 10.8.10.1 netmask 255.255.255.255 destination 10.8.10.2
inet6 fe80::3d6a:b4a:401c:8f5c prefixlen 64 scopeid 0x20<link>
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3 bytes 144 (144.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

tun1: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 10.8.0.1 netmask 255.255.255.255 destination 10.8.0.2
inet6 fe80::dbe1:8cd0:266:be2f prefixlen 64 scopeid 0x20<link>
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3 bytes 144 (144.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Nick Howitt wrote:

Can you give me the output of:

ifconfig

Also have you tried the latest kmod-e1000e driver available from here. You will need to reboot afterwards.

Bane asked "if it is sync or async mode?"

What are the specs for your fibre connection from your ISP - are the upload and download max line rates the same (synchronous), or is the upload a fraction of the download (asynchronous). I suspect from your results the later. When the rates are the same the same bit clock is used for transmission in both directions and hence timing is critical. With asynchronous transmission each direction has its own local independant clock. An example is ADSL where the first letter "A" stands for asynchronous with the upload speed typically 5+% of the download. For instance, my ADSL2+ is 1.14 Mbit/s up and 20.7 Mbit/s down. The synchronous version is SDSL. With cable you will see the term "ATM". Again the first letter is denoting asynchronous.

The J1900 is a slow SoC (System on a Chip). I have a board with the J1800 which is used as a firewall - Multi-Wan with cable and ADSL2+ internet connections. It is quite capable of 1G ethernet as you have found. What I do not know is what additional CPU over head (if any) PPPoE entails. PPPoE can use compression and/or encryption - and if that occurs the slow J1900 may struggle? Run "top" and press the "1" key to see the load on each CPU. Are one or more CPUs running close to 100% when running PPPoE with the modem bridged and running the speed test? How does CPU utilization compare when not bridged?

Did you check the MTU as I described previously - and reduced it to 1492 or a little less if is was set to 1500, to prevent fragmentation?

I thought you were talking about a setting in the PPPoE software on clear. My connection is synchronous, 1gig ( 1 gb up and 1gb down).

Here is a top while running PPPoE and doing a speed test.

op - 14:38:15 up 3:44, 2 users, load average: 0.53, 0.41, 0.24
Tasks: 165 total, 3 running, 162 sleeping, 0 stopped, 0 zombie
%Cpu0 : 15.9 us, 18.5 sy, 0.0 ni, 39.2 id, 0.0 wa, 0.0 hi, 26.4 si, 0.0 st
%Cpu1 : 8.2 us, 7.5 sy, 0.0 ni, 82.6 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu2 : 11.0 us, 22.7 sy, 0.0 ni, 53.5 id, 0.0 wa, 0.0 hi, 12.7 si, 0.0 st
%Cpu3 : 6.6 us, 16.1 sy, 0.0 ni, 69.4 id, 0.0 wa, 0.0 hi, 7.9 si, 0.0 st
KiB Mem : 8081976 total, 6048784 free, 1159764 used, 873428 buff/cache
KiB Swap: 6254588 total, 6254588 free, 0 used. 6596256 avail Mem

Tony Ellis wrote:

Bane asked "if it is sync or async mode?"

What are the specs for your fibre connection from your ISP - are the upload and download max line rates the same (synchronous), or is the upload a fraction of the download (asynchronous). I suspect from your results the later. When the rates are the same the same bit clock is used for transmission in both directions and hence timing is critical. With asynchronous transmission each direction has its own local independant clock. An example is ADSL where the first letter "A" stands for asynchronous with the upload speed typically 5+% of the download. For instance, my ADSL2+ is 1.14 Mbit/s up and 20.7 Mbit/s down. The synchronous version is SDSL. With cable you will see the term "ATM". Again the first letter is denoting asynchronous.

The J1900 is a slow SoC (System on a Chip). I have a board with the J1800 which is used as a firewall - Multi-Wan with cable and ADSL2+ internet connections. It is quite capable of 1G ethernet as you have found. What I do not know is what additional CPU over head (if any) PPPoE entails. PPPoE can use compression and/or encryption - and if that occurs the slow J1900 may struggle? Run "top" and press the "1" key to see the load on each CPU. Are one or more CPUs running close to 100% when running PPPoE with the modem bridged and running the speed test? How does CPU utilization compare when not bridged?

Did you check the MTU as I described previously - and reduced it to 1492 or a little less if is was set to 1500, to prevent fragmentation?

The centurylink supploed C300Z has an MTU of 1492, but setting that on ClearOS made no difference.

Bane wrote:

I thought you were talking about a setting in the PPPoE software on clear. My connection is synchronous, 1gig ( 1 gb up and 1gb down).

Here is a top while running PPPoE and doing a speed test.

op - 14:38:15 up 3:44, 2 users, load average: 0.53, 0.41, 0.24
Tasks: 165 total, 3 running, 162 sleeping, 0 stopped, 0 zombie
%Cpu0 : 15.9 us, 18.5 sy, 0.0 ni, 39.2 id, 0.0 wa, 0.0 hi, 26.4 si, 0.0 st
%Cpu1 : 8.2 us, 7.5 sy, 0.0 ni, 82.6 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu2 : 11.0 us, 22.7 sy, 0.0 ni, 53.5 id, 0.0 wa, 0.0 hi, 12.7 si, 0.0 st
%Cpu3 : 6.6 us, 16.1 sy, 0.0 ni, 69.4 id, 0.0 wa, 0.0 hi, 7.9 si, 0.0 st
KiB Mem : 8081976 total, 6048784 free, 1159764 used, 873428 buff/cache
KiB Swap: 6254588 total, 6254588 free, 0 used. 6596256 avail Mem

Tony Ellis wrote:

Bane asked "if it is sync or async mode?"

What are the specs for your fibre connection from your ISP - are the upload and download max line rates the same (synchronous), or is the upload a fraction of the download (asynchronous). I suspect from your results the later. When the rates are the same the same bit clock is used for transmission in both directions and hence timing is critical. With asynchronous transmission each direction has its own local independant clock. An example is ADSL where the first letter "A" stands for asynchronous with the upload speed typically 5+% of the download. For instance, my ADSL2+ is 1.14 Mbit/s up and 20.7 Mbit/s down. The synchronous version is SDSL. With cable you will see the term "ATM". Again the first letter is denoting asynchronous.

The J1900 is a slow SoC (System on a Chip). I have a board with the J1800 which is used as a firewall - Multi-Wan with cable and ADSL2+ internet connections. It is quite capable of 1G ethernet as you have found. What I do not know is what additional CPU over head (if any) PPPoE entails. PPPoE can use compression and/or encryption - and if that occurs the slow J1900 may struggle? Run "top" and press the "1" key to see the load on each CPU. Are one or more CPUs running close to 100% when running PPPoE with the modem bridged and running the speed test? How does CPU utilization compare when not bridged?

Did you check the MTU as I described previously - and reduced it to 1492 or a little less if is was set to 1500, to prevent fragmentation?

Tony Ellis wrote:

Bane asked "if it is sync or async mode?"

What are the specs for your fibre connection from your ISP - are the upload and download max line rates the same (synchronous), or is the upload a fraction of the download (asynchronous). I suspect from your results the later. When the rates are the same the same bit clock is used for transmission in both directions and hence timing is critical. With asynchronous transmission each direction has its own local independant clock. An example is ADSL where the first letter "A" stands for asynchronous with the upload speed typically 5+% of the download. For instance, my ADSL2+ is 1.14 Mbit/s up and 20.7 Mbit/s down. The synchronous version is SDSL. With cable you will see the term "ATM". Again the first letter is denoting asynchronous.

The J1900 is a slow SoC (System on a Chip). I have a board with the J1800 which is used as a firewall - Multi-Wan with cable and ADSL2+ internet connections. It is quite capable of 1G ethernet as you have found. What I do not know is what additional CPU over head (if any) PPPoE entails. PPPoE can use compression and/or encryption - and if that occurs the slow J1900 may struggle? Run "top" and press the "1" key to see the load on each CPU. Are one or more CPUs running close to 100% when running PPPoE with the modem bridged and running the speed test? How does CPU utilization compare when not bridged?

Did you check the MTU as I described previously - and reduced it to 1492 or a little less if is was set to 1500, to prevent fragmentation?

OK - CPU load and MTU not a problem - sorry, not sure what else to suggest...
Be sure to try the updated driver offered by Nick.

Hmm, so it's a synchronous fibre connection - so even using the modem to do the PPPoE your upload is still way down - 10%, so all is not good even there. Interestingly, the upload is better using the J1900 for PPPoE. Do you happen to know the PCI Express width of the slot the NIC is in - 1x 4x or 8x? My understanding is the Intel 82583V uses the older PCIe v1.0a bus (it was released in 2009 and is a "home" not server product and therefore will not perform the process offloading that the server NICs provide). Just wondering if the bus is a bottleneck. You should be getting close to 1G both ways with a suitable system...

Nick Howitt wrote:
Please can you let me know if this issue is the same as ticket 558584 at Clearcenter?

No it isn't but coincidentally the thread got bumped at about the same time as the ticket after both had been dormant for a while!

It is looking unlikely that the kmod-e1000e driver or even the latest non-kmod one will give you any gain but they are worth trying.

The 7.5 kernel, when it comes out, will come with the same driver as the current kernel, unfortunately. But I believe this kernel comes with other very significant changes which may help if it is not a driver bug but is a kernel bug. I notice the new kernel is available now in the updates-testing repo. I have no idea how safe it is but you can try installing it with a:

yum update kernel --enablerepo=clearos-updates-testing

You will need to reboot for the new kernel to take effect. Note that during boot up it will default to the latest kernel but you get about 5s during the boot to select a different one if necessary. If you try it please report back.

The only other option is to try a USB3 NIC (USB2 has a max speed of 480Mbps). I have absolutely no idea of which USB3 NICs are good in ClearOS.

The D-Link USB 3.0 to Gigabit Ethernet Adapter Model DUB-1312 comes with a tiny CD that has the source for compiling a linux driver compatible with CllearOS 6.x and 7.x. see https://www.dlink.com.au/home-solutions/DUB-1312-usb-3-0-to-gigabit-ethernet-adapter
I have two of these and use them on Mini-ITX Atom boards. The compiled driver is installed at /usr/lib/modules/3.10.0-693.17.1.v7.x86_64/kernel/drivers/net/usb/DUB-13X2.ko
The J1900 would be faster than my Atoms - but would be surprised if you got much over 400 -> 500 Mbit/s - even using a USB3 port. Using iperf with the USB NIC on a USB3 port and connected to a r8168 1 Gbit/s NIC for the results below.

[root@may ~]# iperf -c 192.168.0.16

------------------------------------------------------------

Client connecting to 192.168.0.16, TCP port 5001

TCP window size: 86.2 KByte (default)

------------------------------------------------------------

[  3] local 192.168.0.34 port 35226 connected with 192.168.0.16 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec   332 MBytes   279 Mbits/sec



[root@alice ~]# iperf -c 192.168.0.16

------------------------------------------------------------

Client connecting to 192.168.0.16, TCP port 5001

TCP window size: 86.2 KByte (default)

------------------------------------------------------------

[  3] local 192.168.0.14 port 49172 connected with 192.168.0.16 port 5001

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec   294 MBytes   246 Mbits/sec



[root@may ~]# ethtool enp2s0u1 (one of the USB NICs)

Settings for enp2s0u1:

	Supported ports: [ TP MII ]

	Supported link modes:   10baseT/Half 10baseT/Full 

	                        100baseT/Half 100baseT/Full 

	                        1000baseT/Half 1000baseT/Full 

	Supported pause frame use: No

	Supports auto-negotiation: Yes

	Advertised link modes:  10baseT/Half 10baseT/Full 

	                        100baseT/Half 100baseT/Full 

	                        1000baseT/Full 

	Advertised pause frame use: Symmetric

	Advertised auto-negotiation: Yes

	Link partner advertised link modes:  10baseT/Half 10baseT/Full 

	                                     100baseT/Half 100baseT/Full 

	                                     1000baseT/Half 1000baseT/Full 

	Link partner advertised pause frame use: Symmetric Receive-only

	Link partner advertised auto-negotiation: Yes

	Speed: 1000Mb/s

	Duplex: Full

	Port: MII

	PHYAD: 3

	Transceiver: internal

	Auto-negotiation: on

	Supports Wake-on: pg

	Wake-on: g

	Current message level: 0x00000007 (7)

			       drv probe link

	Link detected: yes

Update: ClearOS 7.5 with kernel 3.10.0-862.6.3.v7.x86_64 now supports the D-Link USB 3.0 to Gigabit Ethernet Adapter Model DUB-1312 using the included ax88179_178a driver...

If it were my machine, I'd probably take a flyer on the new kernel, but that's me. It is also the cheaper and immediate option.

I have seen reports of a StarTech dual USB3 NIC performing at full 1Gb speeds on another distro. Unfortunately in ClearOS it only got up to about 350Mbps.

Hmm, the new updates-testing does not have the required version of linux-firmware so, to install the new kernel:

yum update kernel --enablerepo=clearos-updates-testing,centos-unverified

From the testing I've been seeing, the latest kernel will probably not help, but I have discovered kernel mode PPPoE. Add the following line to your /etc/sysconfig/network-scripts/ifcfg-ppp0 file:

LINUX_PLUGIN=/usr/lib64/pppd/2.4.5/rp-pppoe.so

Then you can try restarting the PPPoE interface. Check that it has worked by doing "ps aux | grep rp-pppoe". If it finds a looooooong line including a reference to /usr/lib64/pppd/2.4.5/rp-pppoe.so then it is using kernel mode. If it does not find it then either restart networking or reboot. On a Core-i5 machine speeds went up from about 290Mbps to c.790Mbps when testing externally but webconfig consumes reasonable resources when testing using the Webconfig widgit. Better would be to test to a PC behind.

IDS had quite an effect on speeds and note also that the proxy/content filter and Application/Protocol filter can also have an impact. If you have the Application or Protocol filter installed you may need to do a "yum remove netifyd" to disable them completely, including from the firewall.

Please see additional comments on Bug 20251. When adding a custom firewall rule, change the "iptables" to "$IPTABLES" and note the alternative way of invoking the plugin which may be better as the webconfig won't overwrite your changes. If you use the alternative way, you may need to use the full path to the plugin.

Please can you post back with your findings?

I did some initial testing, and the performance has increased. The interesting part is that it is now sitting at around 540 down and 900 up. This is with proxy, filtering, and IDS off. So it has improved quite a bit, but not close to my standard century link router which sees 938 down and 900 up.

I don't see a way to make the change permanent. After a reboot, it appears to have been erased out of the ifcfg-ppp0 file. The only thing I see as far as making it permanent is the firewall rule being added via the Web GUI. Is there anyway to add the ifcfg line in the web GUI?

Are there any other tweaks I can make to clear? The CPU doesn't seem to be taxed, and memory is sitting at about 1.6 gb in use of 8gb.

Thanks,

Nick Howitt wrote:

Please see additional comments on Bug 20251. When adding a custom firewall rule, change the "iptables" to "$IPTABLES" and note the alternative way of invoking the plugin which may be better as the webconfig won't overwrite your changes. If you use the alternative way, you may need to use the full path to the plugin.

Please can you post back with your findings?

As you say, the custom firewall rule will make it permanent but use the form "$IPTABLES....." and not "iptables.....".

For the other edit, as mentioned in the bug report, try adding a line to /etc/ppp/options:

plugin rp-pppoe.so

This way the plugin is called from the options file instead. If it does not work, you may need the full path to the plugin, but please report back which works.

There is one further thing to play with and that is the setting SYNCHRONOUS in the ifcfg-pppX file. Try setting it to "yes" and save the file. The interface should restart when you save the file. If it works, to make the change permanent you'll need to set the immutable bit on the file ("chattr +i /etc/sysconfig/network-scripts/ifcfg-ppp0") but then you need to remember you've done it and unset it any time you want a change to the file (for example if you delete the interface through the webconfig to go back to an Ethernet interface)

Hmm. Reading the man pages enabling the plugin disables the SYNCHRONOUS setting so changing it should have no effect. I don't know what else to suggest.

Comparing a DD-WRT set up which I bumped into to ours, the only real difference is they also set the following parameters:

noccp nobsdcomp nomppe nomppc

Doing a "man pppd", I don't see the nomppc option is valid for us and nomppe is the default.You could try setting the other two values in the /etc/ppp/options file. I don't even know if the DD-WRT setup will handle these speeds, but it could be hardware dependant.

Adding noccp and nobsdcomp seems to have increased my bandwidth to 650 down. Upload still hits 940, which is as much as I can expect from upload. I am confused by the difference, especially since I have IDS/IPS, attack detector, proxy, aand content filter turned off.

Also, I am seeing ClearOS fail to give me a result for some websites even though content filtering is off and they work with my CenturyLink router. Interestingly one of the sites is clearos.com, others are missionfed.com and aquasana.com

Nick Howitt wrote:

Comparing a DD-WRT set up which I bumped into to ours, the only real difference is they also set the following parameters:

noccp nobsdcomp nomppe nomppc

Doing a "man pppd", I don't see the nomppc option is valid for us and nomppe is the default.You could try setting the other two values in the /etc/ppp/options file. I don't even know if the DD-WRT setup will handle these speeds, but it could be hardware dependant.

Do you still have any gateway anti-virus running? There isn't a disable button, but you can try uninstalling it - it will take out a few other packages (content filter, clamav) but don't worry while testing. Greater upload speeds than download speeds suggest to me more traffic processing on inbound traffic.

I am surprised those noccp and nobsdcomp parameters make much difference as they tell PPPE not to use different forms of compression, but if they do, it is a gain.

Sites failing to load could be an MTU issue. You could perhaps do some MTU testing. There is a classic ping test which you'll need to google for, or just try setting the MTU to something like 1400 or 1350 and use some trial and error.

Uninstalling, gateway anti-virus, and all of the IDS/IPS/Attack detector packages didn't make much of a difference. I however, found that uninstalling QoS/Bandwidth manager and Directory server netted me almost 100mb more bandwidth. Bandwidth manager was around 80mbps additional back to my download and Directory server added 10mbps.

Looking at top, it looks to me that the majority of the processes never hit more than core 1 of my 4 core CPU. It looks like it could be a case of more multi threading development work being needed on the processes.

I tested using ping and fragment size of 1464 is where it stops erroring out. So 1464 + 28 for headers equals 1492, which is what Centurylink told me to set it to. I set the MTU for PPPoe to 1464, and that fixed the web sites not working issue, but killed the speed tests. With MTU set to 1464, Download is down to 130mbps and up is at 173.

Nick Howitt wrote:

Do you still have any gateway anti-virus running? There isn't a disable button, but you can try uninstalling it - it will take out a few other packages (content filter, clamav) but don't worry while testing. Greater upload speeds than download speeds suggest to me more traffic processing on inbound traffic.

I am surprised those noccp and nobsdcomp parameters make much difference as they tell PPPE not to use different forms of compression, but if they do, it is a gain.

Sites failing to load could be an MTU issue. You could perhaps do some MTU testing. There is a classic ping test which you'll need to google for, or just try setting the MTU to something like 1400 or 1350 and use some trial and error.