bootconf=#conf-base-r4pro-8x#ov-pro-sd#ov-pro-cn13#ov-pro-cn14
HW-acceleration for ending traffic. For routed traffic PPE is needed and hw offloading in nftables. Bridged traffic may need addional changes (openwrt use bridger utility).
(Receive Side Scaling)
using 4 additional IRQ for spreading load
cat /proc/interrupts | grep ethernet
echo 1 > /proc/irq/105/smp_affinity echo 2 > /proc/irq/106/smp_affinity echo 4 > /proc/irq/107/smp_affinity echo 8 > /proc/irq/108/smp_affinity
moving tx frame-engine irq to different cpu (here 3rd) echo 4 > /proc/irq/103/smp_affinity
disable RPS (Receive Packet Steering) for all macs: echo 0 > /sys/devices/platform/soc/15100000.ethernet/net/eth0/queues/rx-0/rps_cpus
pay attention on iperf-version (iperf 3.17 is ok, 3.12 is not)
traffic must be created using multiple streams so that it can be splitted, so use multithreaded iperf3
on R4: bin/iperf3 -s
on the other side: iperf3 -c 192.168.1.1 -i 1 -P 4
you should reach ~9.3 GBit/s
and see spreading load over CPU cores
root@bpi-r4-phy-8G:~# cat /proc/interrupts | grep eth 103: 20 198366 0 0 GICv3 229 Level 15100000.ethernet 105: 3611 0 0 0 GICv3 221 Level 15100000.ethernet, 15100000.ethernet 106: 2 6842 0 0 GICv3 222 Level 15100000.ethernet, 15100000.ethernet 107: 4 0 27643 0 GICv3 223 Level 15100000.ethernet, 15100000.ethernet 108: 3 0 0 27925 GICv3 224 Level 15100000.ethernet, 15100000.ethernet
using the default iperf3 from debian bookworm (3.12) results in only 6.7GBit/s
(Large Receive Offload)
Add HW LRO RX rule:
ethtool -N [interface] flow-type tcp4 dst-ip [IP] action 0 loc [0/1]
Delete HW LRO RX rule:
ethtool -N [interface] delete [0/1]
Enable/Disable HW LRO rule:
ethtool -K [interface] lro [on | off]
Show the current offload features:
ethtool -k [interface]
example:
ethtool -N eth2 flow-type tcp4 dst-ip 192.168.1.1 action 0 loc 0 ethtool -K eth2 lro on ethtool -k eth2
using iperf instead of iperf3!
verify with propritary debugfs Enable HW LRO rings echo 4 1 > /proc/mtketh/hw_lro_auto_tlb Enable HW LRO statistics echo 5 1 > /proc/mtketh/hw_lro_auto_tlb
cat /proc/mtketh/hw_lro_stats