draft-ietf-nvo3-vmm-11.txt   draft-ietf-nvo3-vmm-12.txt 
skipping to change at page 1, line 15 skipping to change at page 1, line 15
Expires: September 30, 2020 Denpel Informatique Expires: September 30, 2020 Denpel Informatique
B.Khasnabish B.Khasnabish
Independent Independent
T. Herbert T. Herbert
Intel Intel
S. Dikshit S. Dikshit
Aruba-HPE Aruba-HPE
March 30, 2020 March 30, 2020
Virtual Machine Mobility Solutions for L2 and L3 Overlay Networks Virtual Machine Mobility Solutions for L2 and L3 Overlay Networks
draft-ietf-nvo3-vmm-11 draft-ietf-nvo3-vmm-12
Abstract Abstract
This document describes virtual machine mobility solutions commonly This document describes virtual machine mobility solutions commonly
used in data centers built with overlay-based network. This document used in data centers built with overlay-based network. This document
is intended for describing the solutions and the impact of moving is intended for describing the solutions and the impact of moving
VMs (or applications) from one Rack to another connected by the VMs (or applications) from one Rack to another connected by the
Overlay networks. Overlay networks.
For layer 2, it is based on using an NVA (Network Virtualization For layer 2, it is based on using an NVA (Network Virtualization
skipping to change at page 2, line 21 skipping to change at page 2, line 21
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on September 27, 2020. This Internet-Draft will expire on September 30, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 44 skipping to change at page 2, line 44
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................3 1. Introduction...................................................3
2. Conventions used in this document..............................4 2. Conventions used in this document..............................4
3. Requirements...................................................5 3. Requirements...................................................5
4. Overview of the VM Mobility Solutions..........................6 4. Overview of the VM Mobility Solutions..........................6
4.1. Inter-VNs communication...................................6 4.1. Inter-VNs and External Communication......................6
4.2. VM Migration in Layer 2 Network...........................6 4.2. VM Migration in Layer 2 Network...........................6
4.3. VM Migration in Layer-3 Network...........................8 4.3. VM Migration in Layer-3 Network...........................8
4.4. Address and Connection Management in VM Migration.........9 4.4. Address and Connection Management in VM Migration.........9
5. Handling Packets in Flight....................................10 5. Handling Packets in Flight....................................10
6. Moving Local State of VM......................................11 6. Moving Local State of VM......................................11
7. Handling of Hot, Warm and Cold VM Mobility....................11 7. Handling of Hot, Warm and Cold VM Mobility....................11
8. Other Options.................................................12 8. Other Options.................................................12
9. VM Lifecycle Management.......................................13 9. VM Lifecycle Management.......................................13
10. Security Considerations......................................13 10. Security Considerations......................................13
11. IANA Considerations..........................................13 11. IANA Considerations..........................................14
12. Acknowledgments..............................................14 12. Acknowledgments..............................................14
13. Change Log...................................................14 13. Change Log...................................................14
14. References...................................................14 14. References...................................................14
14.1. Normative References....................................14 14.1. Normative References....................................15
14.2. Informative References..................................16 14.2. Informative References..................................16
1. Introduction 1. Introduction
This document describes the overlay-based data center networks This document describes the overlay-based data center networks
solutions in supporting multitenancy and VM (Virtual Machine) solutions in supporting multitenancy and VM (Virtual Machine)
mobility. Being able to move VMs dynamically, from one server to mobility. Being able to move VMs dynamically, from one server to
another, makes it possible for dynamic load balancing or work another, makes it possible for dynamic load balancing or work
distribution. Therefore, dynamic VM Mobility is highly desirable distribution. Therefore, dynamic VM Mobility is highly desirable
for large scale multi-tenant DCs. for large scale multi-tenant DCs.
This document is strictly within the DCVPN, as defined by the NVO3 This document is strictly within the DCVPN, as defined by the NVO3
skipping to change at page 6, line 16 skipping to change at page 6, line 16
except for handling packets in flight. except for handling packets in flight.
VM mobility solutions/procedures should not need to use tunneling VM mobility solutions/procedures should not need to use tunneling
except for handling packets in flight. except for handling packets in flight.
4. Overview of the VM Mobility Solutions 4. Overview of the VM Mobility Solutions
Layer 2 and Layer 3 mobility solutions are described respectively Layer 2 and Layer 3 mobility solutions are described respectively
in the following sections. in the following sections.
4.1. Inter-VNs communication 4.1. Inter-VNs and External Communication
Inter VNs (Virtual Networks) communication refers to communication Inter VNs (Virtual Networks) communication refers to communication
among tenants (or hosts) belonging to different VNs. Those tenants among tenants (or hosts) belonging to different VNs. Those tenants
can be attached to the NVEs co-located in the same Data Center or can be attached to the NVEs co-located in the same Data Center or
in different Data centers. This document assumes that the inter- in different Data centers. When a VM communicates with an external
VNs communication is via the NVO3 Gateway as described in RFC8014 entity, the VM is effectively communicating with a peer in a
(NVO3 Architecture). RFC 8014 (Section 5.3) describes the NVO3 different network or a globally reachable host.
Gateway function which is to relay traffic onto and off of a
virtual network, i.e. among different VNs.
When a VM communicates with an external entity, the VM is This document assumes that the inter-VNs communication and the
effectively communicating with a peer in a different network or a communication with external entities are via the NVO3 Gateway as
globally reachable host. Communicating with hosts in other VNs described in RFC8014 (NVO3 Architecture). RFC 8014 (Section 5.3)
and external hosts are all through the NVO3 Gateway. There are describes the NVO3 Gateway function which is to relay traffic onto
different policies on the NVo3 Gateway to govern the communication and off of a virtual network, i.e. among different VNs.
among VNs and with external hosts.
There are different policies on the NVO3 Gateway to govern the
communication among VNs and with external entities (or hosts).
After a VM is moved to a new NVE, the VM's corresponding Gateway After a VM is moved to a new NVE, the VM's corresponding Gateway
may need to change as well. If such a change is not possible, then may need to change as well. If such a change is not possible, then
the path to the external entity need to be hair-pinned to the NVO3 the path to the external entities need to be hair-pinned to the
Gateway used prior to the VM move. NVO3 Gateway used prior to the VM move.
4.2. VM Migration in Layer 2 Network 4.2. VM Migration in Layer 2 Network
In a Layer-2 based approach, VM moving to another NVE does not In a Layer-2 based approach, VM moving to another NVE does not
change its IP address. But this VM is now under a new NVE, change its IP address. But this VM is now under a new NVE,
previously communicating NVEs may continue sending their packets previously communicating NVEs may continue sending their packets
to the Old NVE. Therefore, Address Resolution Protocol (ARP) to the Old NVE. Therefore, Address Resolution Protocol (ARP)
cache in IPv4 [RFC0826] or neighbor cache in IPv6 [RFC4861] in the cache in IPv4 [RFC0826] or neighbor cache in IPv6 [RFC4861] in the
NVEs that have attached VMs communicating with the VM being moved NVEs that have attached VMs communicating with the VM being moved
need to be updated promptly. If the VM being moved has need to be updated promptly. If the VM being moved has
skipping to change at page 12, line 18 skipping to change at page 12, line 18
8. Other Options 8. Other Options
VM Hot mobility is to enable uninterrupted running of the VM Hot mobility is to enable uninterrupted running of the
application or workload instantiated on the VM when the VM running application or workload instantiated on the VM when the VM running
conditions changes, such as utilization overload, hardware running conditions changes, such as utilization overload, hardware running
condition changes, or others. Hot, Warm and Cold mobility are condition changes, or others. Hot, Warm and Cold mobility are
planned activities which are managed by VM management system. planned activities which are managed by VM management system.
For unexpected events, such as unexpected failure, a VM might need For unexpected events, such as unexpected failure, a VM might need
to move to a new NVE, which is called Hot VM Failover in this to move to a new NVE, which is called Hot VM Failover in this
document. For Hot VM Failover, there are VMs in both primary and document. For Hot VM Failover, there are redundant primary and
secondary NVEs. They can provide services simultaneously as in secondary VMs whose states are synchronized by means that are
load-share mode of operation. If the VM in the primary NVE fails, outside the scope of this draft. If the VM in the primary NVE
there is no need to actively move the VM to the secondary NVE fails, there is no need to actively move the VM to the secondary
because the VM in the secondary NVE can immediately pick up the NVE because the VM in the secondary NVE can immediately pick up
processing. It is out of the scope of this document on how and the processing. Details of how state is synchronized between the
what information are exchange between the two VMs under two primary and secondary VMs are beyond the scope of this document.
different NVE.
The VM Failover to the new NVE is transparent to the peers that The VM Failover to the new NVE is transparent to the peers that
communicate with this VM. This can be achieved by both active VM communicate with this VM. This can be achieved by both active VM
and standby VM share the same TCP port and same IP address. There and standby VM share the same TCP port and same IP address, and
must be a load balancer that can distribute the packets to the VM using distributed load balancing functionality that controls which
under the new NVE. The new VM can pick up providing service while VM responds to each service request. In the absence of a failure,
the sender (peer) still continues to receive Ack from the old VM the new VM can pick up providing service while the sender (peer)
and chooses not to use the service of the secondary responding VM. still continues to receive Ack from the old VM. If the situation
If the situation (loading condition of the primary responding VM) (loading condition of the primary responding VM) changes the
changes the secondary responding VM may start providing service to secondary responding VM may start providing service to the sender
the sender (peers). (peers). On failure, the sender (peer) may have to retry the
request, so this structure is limited to requests that can be
safely retried.
If TCP states are not properly synchronized among the two VMs, the If load balancing functionality is not used, the VM Failover can
VM under the New NVE after failover can force the peers to re- be made transparent to the sender (peers) without relying on
establish a new TCP connection by stopping the previous TCP request retry by using techniques described in section 4 that do
connection. As most TCP connections are short lived, re- not depend on the primary VM or its associated NVE doing anything
establishing a new one is not a big problem. after the failure. This restriction is necessary because a
failure that affects the primary VM may also cause its associated
NVE to fail (e.g., if the NVE is located in the hypervisor
that hosts the primary VM and the underlying physical server
fails, both the primary VM and the hypervisor that contains the
NVE fail as a consequence).
The Hot VM Failover option is the costliest mechanism, and hence The Hot VM Failover option is the costliest mechanism, and hence
this option is utilized only for mission-critical applications and this option is utilized only for mission-critical applications and
services. services.
9. VM Lifecycle Management 9. VM Lifecycle Management
The VM lifecycle management is a complicated task, which is beyond The VM lifecycle management is a complicated task, which is beyond
the scope of this document. Not only it involves monitoring server the scope of this document. Not only it involves monitoring server
utilization, balanced distribution of workload, etc., but also utilization, balanced distribution of workload, etc., but also
needs to manage seamlessly VM migration from one server to needs to manage seamlessly VM migration from one server to
another. another.
10. Security Considerations 10. Security Considerations
Security threats for the data and control plane for overlay Security threats for the data and control plane for overlay
networks are discussed in [RFC8014]. ARP (IPv40 and ND (IPv6) are networks are discussed in [RFC8014]. ARP (IPv4) and ND (IPv6) are
not secure, especially if we accept gratuitous versions in multi- not secure, especially if they can be sent gratuitously across
tenant environment. tenant boundaries in a multi-tenant environment.
In Layer-3 based overlay data center networks, ARP and ND messages In overlay data center networks, ARP and ND messages can be used
can be used to mount address spoofing attacks. An NVE may have to mount address spoofing attacks from untrusted VMs and/or other
untrusted VMs attached. This usually happens in cases like the VMs untrusted sources. Examples of untrusted VMs include running third
running third party applications. Those untrusted VMs can send party applications (i.e., applications not written by the tenant
falsified ARP (IPv4) and ND (IPv6) messages, causing NVE, NVO3 who controls the VM). Those untrusted VMs can send falsified ARP
Gateway, and NVA to be overwhelmed and not able to perform (IPv4) and ND (IPv6) messages, causing NVE, NVO3 Gateway, and NVA
legitimate functions. The attacker can intercept, modify, or even to be overwhelmed and not able to perform legitimate functions.
stop data in-transit ARP/ND messages intended for other VNs and The attacker can intercept, modify, or even stop data in-transit
initiate DDOS attacks to other VMs attached to the same NVE. A ARP/ND messages intended for other VNs and initiate DDOS attacks
simple black-hole attacks can be mounted by sending a falsified to other VMs attached to the same NVE. A simple black-hole attacks
ARP/ND message to indicate that the victim's IP address has moved can be mounted by sending a falsified ARP/ND message to indicate
to the attacker's VM. That technique can also be used to mount that the victim's IP address has moved to the attacker's VM. That
man-in-the-middle attacks with some more effort to ensure that the technique can also be used to mount man-in-the-middle attacks with
intercepted traffic is eventually delivered to the victim. some more effort to ensure that the intercepted traffic is
eventually delivered to the victim.
The locator-identifier mechanism given as an example (ILA) doesn't The locator-identifier mechanism given as an example (ILA) doesn't
include secure binding. It doesn't discuss how to securely bind include secure binding. It doesn't discuss how to securely bind
the new locator to the identifier. the new locator to the identifier.
Because of those threats, VM management system needs to apply Because of those threats, VM management system needs to apply
stronger security mechanisms when add a VM to an NVE. Some tenants stronger security mechanisms when add a VM to an NVE. Some tenants
may have requirement that prohibit their VMs to be co-attached to may have requirement that prohibit their VMs to be co-attached to
the NVEs with other tenants. Some Data Centers have their NVO3 the NVEs with other tenants. Some Data Centers have their NVO3
Gateways to be equipped with capability to mitigate ARP/ND Gateways to be equipped with capability to mitigate ARP/ND
threats, such as periodically exchanging its ARP/ND cache with threats, such as periodically exchanging its ARP/ND cache with
NVA's central control system. NVA's central control system to validate the ARP/ND cache learned
by the NVE with the VM Management System.
11. IANA Considerations 11. IANA Considerations
This document makes no request to IANA. This document makes no request to IANA.
12. Acknowledgments 12. Acknowledgments
The authors are grateful to Bob Briscoe, David Black, Dave R. The authors are grateful to Bob Briscoe, David Black, Dave R.
Worley, Qiang Zu, Andrew Malis for helpful comments. Worley, Qiang Zu, Andrew Malis for helpful comments.
skipping to change at page 14, line 23 skipping to change at page 14, line 29
. submitted version -01 with these changes: references are updated, . submitted version -01 with these changes: references are updated,
o added packets in flight definition to Section 2 o added packets in flight definition to Section 2
. submitted version -02 with updated address. . submitted version -02 with updated address.
. submitted version -03 to fix the nits. . submitted version -03 to fix the nits.
. submitted version -04 in reference to the WG Last call comments. . submitted version -04 in reference to the WG Last call comments.
. Submitted version - 05, 06, 07, and 08 to address IETF LC comments . Submitted version - 05, 06, 07, 08, 09, 10, 11, 12 to address IETF
from TSV area. LC comments from TSV area.
14. References 14. References
14.1. Normative References 14.1. Normative References
[RFC0826] Plummer, D., "An Ethernet Address Resolution Protocol: Or [RFC0826] Plummer, D., "An Ethernet Address Resolution Protocol: Or
Converting Network Protocol Addresses to 48.bit Ethernet Converting Network Protocol Addresses to 48.bit Ethernet
Address for Transmission on Ethernet Hardware", STD 37, Address for Transmission on Ethernet Hardware", STD 37,
RFC 826, DOI 10.17487/RFC0826, November 1982, RFC 826, DOI 10.17487/RFC0826, November 1982,
<https://www.rfc-editor.org/info/rfc826>. <https://www.rfc-editor.org/info/rfc826>.
[RFC0903] Finlayson, R., Mann, T., Mogul, J., and M. Theimer, "A [RFC0903] Finlayson, R., Mann, T., Mogul, J., and M. Theimer, "A
Reverse Address Resolution Protocol", STD 38, RFC 903, Reverse Address Resolution Protocol", STD 38, RFC 903,
 End of changes. 17 change blocks. 
61 lines changed or deleted 68 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/