0

I'm having a hard time trying to find a solution, and I can't seem to find anywhere on the web exactly what I need.

We need to configure 2 Routers (MOXA EDR 810) to have N redundant links between one another. Unfortunately, some of the links block L2 packets, therefore we cannot use STP / LACP nor proprietary L2 protocols (e.g. MOXA's Turbo Ring v2).

Example diagram

We are trying to find a workaround using OSPF and N VLANs. The solution works when 1 interface is connected, but it seems to start flapping / losing packets when 2+ interfaces are connected, even though the link weights are different.

I am a newbie with OSPF and I am wondering whether the fact that OSPF sees the same Router ID on all available links causes the flapping.

I have been trying to find an answer online, but I am at a loss, therefore I'm here looking for help.

Any reply is appreciated.

Thank you beforehand


EDIT: Additional details, to collect the comments:

  • RSTP is disabled on all ports
  • Each of the N links is assigned a dedicated untagged VLAN

EDIT 2: configuration files

Router #1

! ---------- EDR-810-VPN-2GSFP-T ----------
router ospf 10.0.1.10
 redistribute connected
 redistribute static
 redistribute rip
 area 0.0.0.0
vlan create  1
vlan create  90
vlan create  91
interface ethernet 1/1
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/2
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/3
 no shutdown 
 name PC
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/4
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/5
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/6
 no shutdown 
 name PTP1
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  90
 no spanning-tree edge-port 
interface ethernet 1/7
 no shutdown 
 name PTP2
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  91
 no spanning-tree edge-port 
interface ethernet 1/8
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/9
 no shutdown 
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/10
 no shutdown 
 switchport access vlan  1
 no spanning-tree edge-port 
interface lan 
 bind vlan 1
 ip address  static 10.0.1.10 255.255.255.0
 name  LAN
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface vlan  90
 ip address   10.0.90.1 255.255.255.0
 name  PTP1
 ip ospf area 0.0.0.0
 ip ospf hello-interval 5
 ip ospf dead-interval 10
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface vlan  91
 ip address   10.0.91.1 255.255.255.0
 name  PTP2
 ip ospf area 0.0.0.0
 ip ospf priority 10
 ip ospf hello-interval 5
 ip ospf dead-interval 10
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface bridge 
 ip address 192.168.126.254 255.255.255.0
 name BRG_LAN
 shutdown 
 no goose-pass-through 
interface zone-base-bridge 
 ip address 0.0.0.0 0.0.0.0
 name ZONE_BRG
 shutdown 
 no goose-pass-through 
dot1x reauth period 3600
interface wan 
 shutdown 
 ip address static 10.0.0.2 255.255.255.0 10.0.0.1 
firewall 1
 action drop
 interface ALL ALL
 protocol All
 mode ip
 src-ip all
 src-port all
 dst-ip all
 dst-port all
 logging severity 0
firewall 1 disable
firewall malformed logging severity 0
l2-filter 1
 action accept
 interface all all
 protocol all
 src-mac 00:00:00:00:00:00
 dst-mac 00:00:00:00:00:00
l2-filter 1 disable
dos icmp-death 4000
no dos icmp-death
dos syn-flood 4000
no dos syn-flood
dos arp-flood 4000
no dos arp-flood
logging dos severity 0
snmp-server trap-mode trap-v1
ip dhcp pool 1
 network 10.0.1.100 10.0.1.200 255.255.255.0
 lease 60
 default-router 10.0.1.10
ip ddns service disable
service dhcp 
no lldp enable 
lldp timer 30
hostname MOXA1
snmp-server location ONE
settingcheck timer 180
logging trusted-access severity 0
modbus-filter 1
 action accept
 protocol all
 src-ip all
 dst-ip all
 function 0
 uid 0
 interface all all 
modbus-filter 1 disable
redundancy 
 turbo-ring-v2  1 primary interface 1/6 secondary interface 1/7
 no turbo-ring-v2  1 master
 no turbo-ring-v2  2 master
 no turbo-ring-v2  2
 no turbo-ring-v2 coupling  
redundancy mode rstp
no ip igmp-snooping 
qos mapping dscp-to-queue 30 2
qos mode weighted-fair
monitor source interface 1/6 both
monitor destination interface 1/2
username admin password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 1
username configadmin password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 2
username configadmin privilege 4
username user password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 3
moxa-utility 
ip telnet port 23
ip telnet
ip ssh port 22
ip ssh
ip telnet max-login-users 5
ip http-server port 80
ip http-server
ip http-server secure port 443
ip http-server secure
ip http-server max-login-users 5
no ip ping-response
ip auto-logout 0
auth radius auth-type pap
openvpn server 1
 device-type tap
 protocol udp
 port 1194
 no client-to-client 
 no comp-lzo 
 keepalive 
 cipher BF-CBC
 hash SHA1
 network 10.8.0.0 255.255.255.0
 push route 192.168.127.0 255.255.255.0
 server-bridge 
 bridge-if LAN
openvpn server 1 disable
openvpn client 1
 device-type tun
 protocol udp
 server 0.0.0.0 port 1194
 cipher BF-CBC
 hash SHA1
 comp-lzo 
 client-auth certificate
openvpn client 1 disable
openvpn client 2
 device-type tun
 protocol udp
 server 0.0.0.0 port 1194
 cipher BF-CBC
 hash SHA1
 comp-lzo 
 client-auth certificate
openvpn client 2 disable
no security-notification event-firewall
no security-notification event-dosattack
no security-notification event-accessviolation
no security-notification event-loginfail
password-policy minimum-length 4
no password-policy complexity-check
no password-policy complexity-check digit
no password-policy complexity-check alphabet
no password-policy complexity-check special-characters
no login-lockout 
login-lockout retry-threshold 5
login-lockout lockout-time 5
no logging-capacity 
logging-capacity snmp-trap-warning
logging-capacity email-warning
logging-capacity over-size-action overwrite-oldest
no fast-bootup 
mtu LAN 1500
mtu PTP1 1500
mtu PTP2 1500
web-display-language 0

Router #2

! ---------- EDR-810-VPN-2GSFP-T ----------
router ospf 10.0.2.10
 redistribute connected
 redistribute static
 redistribute rip
 area 0.0.0.0
vlan create  1
vlan create  90
vlan create  91
interface ethernet 1/1
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/2
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/3
 no shutdown 
 name PC
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/4
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/5
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/6
 no shutdown 
 name PTP1
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  90
 no spanning-tree edge-port 
interface ethernet 1/7
 no shutdown 
 name PTP2
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  91
 no spanning-tree edge-port 
interface ethernet 1/8
 no shutdown 
 speed-duplex Auto
 no flowcontrol 
 media cable-mode auto
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/9
 no shutdown 
 switchport access vlan  1
 no spanning-tree edge-port 
interface ethernet 1/10
 no shutdown 
 switchport access vlan  1
 no spanning-tree edge-port 
interface lan 
 bind vlan 1
 ip address  static 10.0.2.10 255.255.255.0
 name  LAN
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface vlan  90
 ip address   10.0.90.2 255.255.255.0
 name  PTP1
 ip ospf area 0.0.0.0
 ip ospf hello-interval 5
 ip ospf dead-interval 10
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface vlan  91
 ip address   10.0.91.2 255.255.255.0
 name  PTP2
 ip ospf area 0.0.0.0
 ip ospf priority 10
 ip ospf hello-interval 5
 ip ospf dead-interval 10
 ip dvmrp 
 ip pim-sm 
 ip pim-sm hello-interval 5
interface bridge 
 ip address 192.168.126.254 255.255.255.0
 name BRG_LAN
 shutdown 
 no goose-pass-through 
interface zone-base-bridge 
 ip address 0.0.0.0 0.0.0.0
 name ZONE_BRG
 shutdown 
 no goose-pass-through 
dot1x reauth period 3600
interface wan 
 shutdown 
 ip address static 10.0.100.2 255.255.255.0 10.0.100.1 
firewall 1
 action drop
 interface ALL ALL
 protocol All
 mode ip
 src-ip all
 src-port all
 dst-ip all
 dst-port all
 logging severity 0
firewall 1 disable
firewall malformed logging severity 0
l2-filter 1
 action accept
 interface all all
 protocol all
 src-mac 00:00:00:00:00:00
 dst-mac 00:00:00:00:00:00
l2-filter 1 disable
dos icmp-death 4000
no dos icmp-death
dos syn-flood 4000
no dos syn-flood
dos arp-flood 4000
no dos arp-flood
logging dos severity 0
snmp-server trap-mode trap-v1
ip dhcp pool 1
 network 10.0.2.100 10.0.2.200 255.255.255.0
 lease 60
 default-router 10.0.2.10
ip ddns service disable
service dhcp 
no lldp enable 
lldp timer 30
hostname MOXA2
snmp-server location TWO
settingcheck timer 180
logging trusted-access severity 0
modbus-filter 1
 action accept
 protocol all
 src-ip all
 dst-ip all
 function 0
 uid 0
 interface all all 
modbus-filter 1 disable
redundancy 
 turbo-ring-v2  1 primary interface 1/6 secondary interface 1/7
 no turbo-ring-v2  1 master
 no turbo-ring-v2  2 master
 no turbo-ring-v2  2
 no turbo-ring-v2 coupling  
redundancy mode rstp
no ip igmp-snooping 
qos mapping dscp-to-queue 30 2
qos mode weighted-fair
username admin password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 1
username configadmin password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 2
username user password 810448e13d53513dddd17d6c045025abddd17d6c045025abddd17d6c045025ab31048d2e6661d3e07ec5d571e37f1886 privilege 3
moxa-utility 
ip telnet port 23
ip telnet
ip ssh port 22
ip ssh
ip telnet max-login-users 5
ip http-server port 80
ip http-server
ip http-server secure port 443
ip http-server secure
ip http-server max-login-users 5
no ip ping-response
ip auto-logout 0
auth radius auth-type pap
openvpn server 1
 device-type tun
 protocol udp
 port 1194
 no client-to-client 
 comp-lzo 
 keepalive 
 cipher BF-CBC
 hash SHA1
 network 10.8.0.0 255.255.255.0
 push route 192.168.127.0 255.255.255.0
 server-bridge 
openvpn server 1 disable
openvpn client 1
 device-type tun
 protocol udp
 server 0.0.0.0 port 1194
 cipher BF-CBC
 hash SHA1
 comp-lzo 
 client-auth certificate
openvpn client 1 disable
openvpn client 2
 device-type tun
 protocol udp
 server 0.0.0.0 port 1194
 cipher BF-CBC
 hash SHA1
 comp-lzo 
 client-auth certificate
openvpn client 2 disable
no security-notification event-firewall
no security-notification event-dosattack
no security-notification event-accessviolation
no security-notification event-loginfail
password-policy minimum-length 4
no password-policy complexity-check
no password-policy complexity-check digit
no password-policy complexity-check alphabet
no password-policy complexity-check special-characters
no login-lockout 
login-lockout retry-threshold 5
login-lockout lockout-time 5
no logging-capacity 
logging-capacity snmp-trap-warning
logging-capacity email-warning
logging-capacity over-size-action overwrite-oldest
no fast-bootup 
mtu LAN 1500
mtu PTP1 1500
mtu PTP2 1500
web-display-language 0

EDIT 3: self answered below with Supplier solution.

17
  • Can you describe these links that block L2 frames? It sounds like you have an L2 loop somewhere. Also, OSPF is not the problem.
    – Ron Trunk
    Apr 6 at 14:43
  • Unfortunately the links for us are blackboxes, we could only verify using Wireshark that STP packets (nor any other L2 packets we tried) pass through. The L2 loop is caused by these "L2 filtered" links, for the moment using VLANs at least each switch local network does not break, but the OSPF is not stable (sorry if not clear) Apr 6 at 14:52
  • When you say "flapping," are the links going up and down? Or are the OSPF adjacencies going up and down?
    – Ron Trunk
    Apr 6 at 14:54
  • I might have used "flapping" wrong, the OSPF adjacencies seem fine, if I have 2 links connected the neighbor state on one router is FULL/DR for both links and FULL/Backup for both links on the other router. Although, if I try to ping end to end some pings are received, other never find a reply (30 to 50% "drop" let's say) Apr 6 at 15:03
  • As a troubleshooting step(s): 1. Turn off or disable OSPF. 2. For each link, ping router 2's link interface from router 1.
    – Ron Trunk
    Apr 6 at 15:15
1

I'm adding an answer to the question as the supplier found the solution to the issue we were experiencing. I'm merely adding here his explanation, it could be useful to anyone else incurring in this event. Thank you again!


[13 April 2021 12:31]

I found the solution that you need to add the Virtual MAC address on one of the interfaces of both EDR.

Please find the attachment of my example, Solution.pptx, the virtual MAC needs to be added on the same VLAN of both EDR-810, e.g. on both VLAN91, add the virtual MAC 00:90:e8:00:00:01 & 02 on each EDR.

With this configuration, the communication between the two laptops will be continued without dropping the packets.

I tested the configuration and it worked totally fine. I expanded the solution to a 3-link OSPF setup, and it scaled correctly. I sent back the configurations asking for further details and they gave a very clear explanation:

[15 April 2021 18:12]

I checked your configurations and they are correct.

This is the correct configuration on the EDR-810 due to EDR-810's switching chip is the SVL (Shared VLAN learning mode)
https://forum.huawei.com/enterprise/en/ivl-svl/thread/607910-875

Where it is explained that

MAC Address Learning Mode includes the IVL mode and SVL mode. IVL, which is the independent VLAN learning mode, indicates that each VLAN can learn the MAC address. In addition, the MAC address table of each VLAN is independent, and the same MAC address can exist in multiple VLANs. SVL, which is the shared VLAN learning mode, indicates that the MAC address learned in a VLAN can be shared with other VLANs, and the MAC address is unique in the MAC address table.

In the IVL mode, each VLAN has its own MAC forwarding table. During the forwarding, the MAC address is found according to the VLAN, and then the forwarding is performed according to the MAC address table. The broadcast packets are forwarded in this VLAN. In the SVL mode, all the VLANs share the same MAC forwarding table. During the forwarding, the egress port is found according to the MAC address, and then the port determines whether this VLAN is allowed to pass.

If the same MAC address is in multiple VLANs, the IVL mode should be used. In this case, if the SVL mode is used, the service cannot differentiate the VLAN.

Applied to our case

MAC address is overwritten in SVL table every time a new OSPF packet arrives Correct routing Failed routing Correct configuration

0

RSTP interferes with your setup of routed links.

As far as I can extract from the manual, the EDR-810 cannot separate its switch ports or configure them as routed (L3) ports.

As a workaround, configure each port with a VLAN of its own, untagged. Then configure each VLAN like a routed port. Disable RSTP on those ports as well - when there's only a single port, there can be no loop. (RSTP blocks ports it deems redundant regardless of VLAN membership. VLAN-aware STP requires MSTP with multiple instances which that device doesn't support.)

That being done, you shouldn't have any more problem configuring multiple ports for redundant routing using OSPF.

[edit] Since there appears to be an unexplained, rather long delay for the test pings (causing them to time out) with otherwise stable routing tables, I'd seriously suspect a bug in the routing engine with multipathing, or in the dynamic update mechanism.

13
  • We have done the solution we proposed (N VLANs for N links, I might have given it for granted, my bad) but it is with this configuration that the issue arises. Each link has a VLAN on its own (I tried to add an image to the post but I can't see it atm), and OSPF is configured on each VLAN. The OSPF adjacency can settle and one router is elected as DR, the other as Backup, but if I try to ping end to end most of the pings never find a reply (30 / 50%) Apr 6 at 15:07
  • Hmm. Do try to ping across the adjacent links. Also, check the routing tables for correctness or flapping gateways.
    – Zac67
    Apr 6 at 15:22
  • Sure, I cannot access the equipment now I will try and add more details tomorrow (like in another comment, there might be some other protocol messing up with the setup like your guess on RTSP, which was as a matter of fact messing with osfp when we tried to turn it on) Apr 6 at 15:43
  • It would also be interesting whether the ping losses are entirely random or in bursts - run with short timeout to get a better picture. Have you checked the logs btw?
    – Zac67
    Apr 6 at 17:10
  • Yes, I just checked, the losses are in bursts, of variable lenghts. I tried to mirror the two ports of the two links (VLAN90 and VLAN91) to a third one to understand if perhaps some pings go through one route and the others go via the other route, causing the loss, but I couldn't make it, I need one PC more to do that Apr 7 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.