VRRP HLD #475

dks19 · 2019-09-30T19:53:16Z

Hi all,
This is high level design for VRRP feature in SONiC. Currently, VRRPv2 along with interface tracking is supported. Please review and provide feedback/comments.

Thanks,
Dilip

msftclas · 2019-09-30T19:53:29Z

All CLA requirements met.

dev-aviz · 2019-10-22T16:42:28Z

doc/SONiC_VRRP_HLD.md

+The Virtual Router Redundancy Protocol (VRRP) functionality is designed to eliminate the single point of
+failure inherent in the static default routed environment. VRRP specifies an election protocol that
+dynamically assigns responsibility of gateway router to a VRRP instance on one of the routers on a LAN. The VRRP instance controlling the IP address(es) associated with a virtual router is called the Master, and routes the traffic. The election process provides dynamic fail-over in the forwarding responsibility should the Master become unavailable. Any of the virtual router's IP addresses on a LAN can then be used as the default first hop router by end-hosts. The advantage gained from using VRRP is a higher availability default path without requiring configuration of dynamic routing or router discovery protocols on every end-host.
+


It would be good to list the out the possible use cases for the sonic user to enable to this feature? Can this work on data center MLAG kind of deployments ?

The use cases are networks needing backup of logical gateway. The network could be like described below or for a similar use case,

3-tier access layer, distribution layer & core layer network, where the redundancy for gateway is needed in distribution layer.

Layer 3 CLOS network, where the redundancy is needed for the first hop gateways.

Generally static anycast gateway is used in MLAG deployment, but yes VRRP could also be used in MLAG deployment.

dev-aviz · 2019-10-22T16:45:10Z

doc/SONiC_VRRP_HLD.md

+
+Following requirements are beyond scope of this release. 
+
+1. VRRPv3 (IPv6) support


what is preventing not to support vrrp3?
How is it different from FRR VRRP? Since FRR is part of SONiC stack, can't we stick to FRR VRRP (one stack)?

Currently VRRPv3 (IPv6) support is added.

Since keepalived VRRP is robust, feature rich and well maintained in linux community, keepalived VRRP was chosen over FRR VRRP.

dev-aviz · 2019-10-22T16:47:14Z

doc/SONiC_VRRP_HLD.md

+
+### 3.2.6   Uplink Interface Tracking
+
+Interfaces other than the VRRP instance interface can be tracked for up/down events. When interface-tracking is enabled in the VRRP instance configuration, the tracked interface's operational status will be monitored. When a interface operational down event is detected on a tracked-interface, the track-priority/weight is subtracted from the current router’s priority value. Similarly, when interface operational up event is detected on the tracked-interface, the track-priority/weight is added to the router’s current priority value.


How does the uplink tracking works? for instance let's say there are more than 8 uplink interfaces how do we does it effects on mastership?

Uplink tracking affects the overall priority for VRRP router. If the tracked interface is in 'UP' state, then the VRRP priority for the router is increased by the weight associated with the tracked interface.

Currently only 8 interfaces can be tracked per VRRP instance. Please let us know if there is a real use case of tracking more than 8 interfaces per VRRP instance.

Note multiple VRRP instances could be configured on an VRRP interface.

lguohan · 2019-10-24T04:54:32Z

doc/SONiC_VRRP_HLD.md

+
+## 7 Warm Reboot Support
+
+Currently, warm-reboot is not supported for VRRP. That is, warm-reboot will simply restart the VRRP docker without VRRP storing any data for warm restart. 


we should prevent warm reboot if there is vrrp configuraiton in place.

In the new design VRRP master's will relinquish the mastership of VRRP instances (by sending priority 0 keepalives, so that the standby can take over the mastership instantaneously) before going to warm-reboot. During warm-reboot the neighboring routers will be master. Post warm-reboot the rebooted router could claim back the mastership.

This way the forwarding is kept functional (and not changing) during the process of warm-reboot.

lguohan · 2019-10-24T04:57:10Z

doc/SONiC_VRRP_HLD.md

+
+### vrrporchd
+
+- Listens to VRRP_Table in APP_DB and adds virtual MAC entries in ASIC_DB for Master instances


it is unclear. can you explain what does vrrporch do?

vrrpsyncd listen to kernel and for each VRRP master instances, vrrpsyncd will add interface name, VIP & VMAC to APP_DB.
vrrporch will inturn listen to APP_DB and for entry in VRRP_TABLE, program the VIP as my IP and the VMAC as my MAC(virtual RIF) in ASIC_DB.

lguohan · 2019-10-24T04:58:51Z

doc/SONiC_VRRP_HLD.md

+between VRID and addresses must be coordinated among all VRRP routers on a LAN. However, there is
+no restriction against reusing a VRID with a different address mapping on different LANs. The scope of
+each virtual router is restricted to a single LAN.
+To minimize network traffic, only the Master for each virtual router sends periodic VRRP Advertisement


break paragraph

Ok, will split it.

lguohan · 2019-10-24T05:02:09Z

doc/SONiC_VRRP_HLD.md

+
+Currently, warm-reboot is not supported for VRRP. That is, warm-reboot will simply restart the VRRP docker without VRRP storing any data for warm restart. 
+
+## 8 Unit Test cases


we need detailed integration test plan

lguohan · 2019-10-24T05:05:33Z

doc/SONiC_VRRP_HLD.md

+
+### 3.2.4    VRRP Advertisement Frame
+
+VRRP control packets have IP protocol type as 112 (reserved for VRRP), and are sent to VRRP multicast address 224.0.0.18. Source MAC in VRRP control packets is virtual MAC address and source IP is interface IP.                                                               


we need sai trap id for this. do we have it defined in the SAI, if yes, then add this as a sai requirement.

The already defined SAI trap IDs viz., SAI_HOSTIF_TRAP_TYPE_VRRP & SAI_HOSTIF_TRAP_TYPE_VRRPV6 will be used to trap the VRRP packets.

lguohan · 2019-10-24T05:07:15Z

doc/SONiC_VRRP_HLD.md

+
+Example:- 
+
+**Key**: VRRP_TABLE:Vlan1000:[40.10.8.101/32:ipv4]


why not use interface table? why can we extend intforch? why do we need a separate vrrporch?

Intforch was not used to separate the use case. In VRRP multiple my IPs and my MACs (virtual RIF) needs to be programmed and it could so happen that there could be multiple sources (different VRRP instances on the same interface), generating same my IP.

As intforch cannot handle it as is, creating vrrporch module to handle the functionality separately.

lguohan · 2019-10-24T05:12:46Z

doc/SONiC_VRRP_HLD.md

+
+### vrrpsyncd
+
+Listens to MACVLAN interface programming in kernel. Status of MACVLAN interface determines Master/Backup state of VRRP instances. VRRP_Table in APP_DB will be programmed with interface name and VIP for Master instances.


macvlan interface can be used for other use cases, are we assuming all macvlan interface are created by keepalived?

No, only MACVLANs whose interface name starts with 'vrrp' in kernel is associated with VRRP. keepalived adds MACVLANs with name starting with 'vrrp' substring.

disaster123 · 2019-10-30T18:35:31Z

Does this one support active/active setups?

dks19 · 2019-10-30T19:58:49Z

This implementation of VRRP doesn't support Active-Active mode.

…

On Wed, Oct 30, 2019 at 11:35 AM disaster123 ***@***.***> wrote: Does this one support active/active setups? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#475>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQHQWHJ4GPBZEV5NGHSEYLQRHHYTANCNFSM4I374EJA> .

disaster123 · 2019-11-05T22:37:29Z

@dks19 any chance to implement active active or vrr? I‘m willing to pay for it.

nser77 · 2023-05-01T19:37:32Z

Quite interesting topic.

I personally find a lot of benefit into keepalived:

keepalived is heavily focused on the vrrp protocol (v2 and v3) and supports many configurations around it.
keepalived has many track controls to detect a failure: interfaces, script, process and files.
keepalived supports both iptables and nftables.

           # keepalived uses a firewall (either nftables or iptables) for two purposes:
           #  i)  To implement no_accept mode
           #  ii) To stop IGMP/MLD/Router-Solicit packets being sent on VMAC interfaces,
           #      and to move IGMP/MLD messages onto the underlying interface.

keepalived supports VRRP synchronization group(s): VRRP Sync Group is an extension to VRRP protocol (man).
keepalived supports static route creation - it may need some implementations like fpmsyncd with bgp (see above).
keepalived is a lightweight, robust and smart software.
keepalived uses SNMP.
keepalived works with namespaces.

Anyway, keepalived works quite close to the kernel so in this case the heavy problem might be related to the netlink integration and how keepalived could works natively into the SONiC environment.

[...]
Same as IPVS wrapper. Keepalived work with its own network interface representation. 
IP address and interface flags are set and monitored through kernel Netlink channel. 
The Netlink messaging sub-system is used for setting VRRP VIPs. 
On the other hand, the Netlink kernel messaging broadcast capability is used to reflect 
into our userspace Keepalived internal data representation any events related to interfaces. 
So any other userspace (others program) netlink manipulation is reflected to our 
Keepalived data representation via Netlink Kernel broadcast (RTMGRP_LINK & RTMGRP_IPV4_IFADDR).
[...]

As per SONiC achitecture, seems to be logic to manage the integration betwen keepalived and netlink trought the producer/consumer achitecture and with some of those processes that are already implemented for some of those scopes:

[...]

Portsyncd: Listens to port-related netlink events. 
During boot-up, portsyncd obtains physical-port information by parsing system's hardware-profile config files.
In all these cases, portsyncd ends up pushing all the collected state into APPL_DB.
Attributes such as port-speeds, lanes and mtu are transferred through this channel.
Portsyncd also inject state into STATE_DB. Refer to next section for more details.

Intfsyncd: Listens to interface-related netlink events and push collected state into APPL_DB. 
Attributes such as new/changed ip-addresses associated to an interface are handled by this process.

[...]

Orchagent: The most critical component in the Swss subsystem. 
Orchagent contains logic to extract all the relevant state injected by *syncd daemons, 
process and message this information accordingly, and finally push it towards its south-bound interface.
This south-bound interface is yet again another database within the redis-engine (ASIC_DB), so as we can see, 
Orchagent operates both as a consumer (for example for state coming from APPL_DB),
and also as a producer (for state being pushed into ASIC_DB).

IntfMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure interfaces in the linux kernel. 
This step is only accomplished if there is no conflicting or inconsistent state within any of the databases being monitored.
Refer to the above database-container section for examples of this undesired behavior.

VlanMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure vlan-interfaces in the linux kernel.
As in IntfMgrd's case, this step will be only attempted if there is no dependent state/conditions being unmet.

[...]

As @dks19 suggested in his HLD, a vrrpsyncd process should also be added and, as per SONiC architecture, it may need to act as a producer for kernel integration (routes, ipaddress, etc. ..) - like fpmsyncd acts for bgp - and also as consumer, for all of those required netlink messages in both directions; from my point of view, the integration betwen keepalived and SONiC should relay over the SAI ecosystem and keepalivedshould not directly operate to the kernel; on the other hand, I'm not sure if vrrporchd is required.

Remember that vrrp works over multicast, but keepalived also implements unicast.

The integration of keepalived in SONiC would be very useful if it were fully integrated into the producer/consumer architecture, that's the power of SONiC, but keepalived really works close to the kernel; so, before proceding, an handshake with the mainters of keepalived might be required.

At the moment, I'm quite interested but still skeptical about this integration (perhaps due to lack of knowledge), anyway, keepalived should not replace frr in the SONiC architecture, but simply provide a mission-critical vrrp support for the enterprise in a separate, dedicated container.

linux-foundation-easycla · 2023-07-14T23:55:04Z

❌ - login: @dks19 / name: Dilip . The commit (0aa1db5, 3ceee3a, 7c77c69, eb48cde, 33c2c86, b13dd68) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.
❌ - login: @vvbrcm . The commit (8a93483, 09f1143, 37b9d0d, bed3582) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

jefftant · 2023-07-15T00:37:37Z

Please note that we (IETF RTGWG chairs) have just last called draft-ietf-rtgwg-vrrp-rfc5798bis, upon becoming RFC it will absolete RFC5798, please use it as a base. Cheers, Jeff

…

On Fri, Jul 14, 2023 at 16:55 linux-foundation-easycla[bot] ***@***.*** ***@***.***>> wrote: <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2> ❌ <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2> - login: @dks19 <https://github.com/dks19> / name: Dilip . The commit (0aa1db5 <0aa1db5>, 3ceee3a <3ceee3a>, 7c77c69 <7c77c69>, eb48cde <eb48cde>, 33c2c86 <33c2c86>) is not authorized under a signed CLA. Please click here to be authorized <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2>. For further assistance with EasyCLA, please submit a support request ticket <https://jira.linuxfoundation.org/servicedesk/customer/portal/4>. — Reply to this email directly, view it on GitHub <#475 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSIDQ6QJZ4YYRNSPIIVE5TXQHL6PANCNFSM4I374EJA>. You are receiving this because you are subscribed to this thread.

Update SONiC_VRRP_HLD.md

adyeung · 2024-04-11T22:32:53Z

Converge VRRP proposal with #1446

dks19 added 3 commits September 30, 2019 12:10

Added VRRP HLD

0aa1db5

Minor update to VRRP HLD

3ceee3a

Corrected links to the diagrams

7c77c69

Updated the diagram

eb48cde

dev-aviz reviewed Oct 22, 2019

View reviewed changes

lguohan reviewed Oct 24, 2019

View reviewed changes

yxieca force-pushed the master branch 2 times, most recently from 8498931 to 8837dc2 Compare April 15, 2022 16:51

Updated the document as per the review comments from community members

33c2c86

prvattem mentioned this pull request Aug 22, 2023

Virtual Router Redundancy Protocol Adaptation HLD #1446

Open

vvbrcm and others added 5 commits August 23, 2023 17:06

Update SONiC_VRRP_HLD.md

8a93483

Update SONiC_VRRP_HLD.md

09f1143

Update SONiC_VRRP_HLD.md

37b9d0d

Add files via upload

bed3582

Merge pull request #1 from vvbrcm/master-1

b13dd68

Update SONiC_VRRP_HLD.md

adyeung closed this Apr 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VRRP HLD #475

VRRP HLD #475

dks19 commented Sep 30, 2019

msftclas commented Sep 30, 2019 •

edited

dev-aviz Oct 22, 2019

vvbrcm Jul 11, 2023

dev-aviz Oct 22, 2019

vvbrcm Jul 11, 2023

dev-aviz Oct 22, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

lguohan Oct 24, 2019

vvbrcm Jul 11, 2023

disaster123 commented Oct 30, 2019

dks19 commented Oct 30, 2019 via email

disaster123 commented Nov 5, 2019

nser77 commented May 1, 2023

linux-foundation-easycla bot commented Jul 14, 2023 •

edited

jefftant commented Jul 15, 2023 via email

adyeung commented Apr 11, 2024


		Following requirements are beyond scope of this release.

		1. VRRPv3 (IPv6) support


		### 3.2.6 Uplink Interface Tracking

		Interfaces other than the VRRP instance interface can be tracked for up/down events. When interface-tracking is enabled in the VRRP instance configuration, the tracked interface's operational status will be monitored. When a interface operational down event is detected on a tracked-interface, the track-priority/weight is subtracted from the current router’s priority value. Similarly, when interface operational up event is detected on the tracked-interface, the track-priority/weight is added to the router’s current priority value.


		## 7 Warm Reboot Support

		Currently, warm-reboot is not supported for VRRP. That is, warm-reboot will simply restart the VRRP docker without VRRP storing any data for warm restart.


		### vrrporchd

		- Listens to VRRP_Table in APP_DB and adds virtual MAC entries in ASIC_DB for Master instances


		Currently, warm-reboot is not supported for VRRP. That is, warm-reboot will simply restart the VRRP docker without VRRP storing any data for warm restart.

		## 8 Unit Test cases


		### 3.2.4 VRRP Advertisement Frame

		VRRP control packets have IP protocol type as 112 (reserved for VRRP), and are sent to VRRP multicast address 224.0.0.18. Source MAC in VRRP control packets is virtual MAC address and source IP is interface IP.


		### vrrpsyncd

		Listens to MACVLAN interface programming in kernel. Status of MACVLAN interface determines Master/Backup state of VRRP instances. VRRP_Table in APP_DB will be programmed with interface name and VIP for Master instances.


		Example:-

		Key: VRRP_TABLE:Vlan1000:[40.10.8.101/32:ipv4]

VRRP HLD #475

VRRP HLD #475

Conversation

dks19 commented Sep 30, 2019

msftclas commented Sep 30, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

disaster123 commented Oct 30, 2019

dks19 commented Oct 30, 2019 via email

disaster123 commented Nov 5, 2019

nser77 commented May 1, 2023

linux-foundation-easycla bot commented Jul 14, 2023 • edited

jefftant commented Jul 15, 2023 via email

adyeung commented Apr 11, 2024

msftclas commented Sep 30, 2019 •

edited

linux-foundation-easycla bot commented Jul 14, 2023 •

edited