--- 1/draft-dt-detnet-dp-alt-03.txt 2016-09-14 04:18:08.887714974 -0700 +++ 2/draft-dt-detnet-dp-alt-04.txt 2016-09-14 04:18:08.999717785 -0700 @@ -1,26 +1,26 @@ DetNet J. Korhonen, Ed. Internet-Draft Broadcom Intended status: Informational J. Farkas -Expires: February 18, 2017 G. Mirsky +Expires: March 18, 2017 G. Mirsky Ericsson P. Thubert Cisco Y. Zhuang Huawei L. Berger LabN - August 17, 2016 + September 14, 2016 DetNet Data Plane Protocol and Solution Alternatives - draft-dt-detnet-dp-alt-03 + draft-dt-detnet-dp-alt-04 Abstract This document identifies existing IP and MPLS, and other encapsulations that run over IP and/or MPLS data plane technologies that can be considered as the base line solution for deterministic networking data plane definition. Status of This Memo @@ -30,21 +30,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on February 18, 2017. + This Internet-Draft will expire on March 18, 2017. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -69,37 +69,36 @@ 4.6. #6 Operations, Administration and Maintenance . . . . . . 11 4.7. #8 Class and quality of service capabilities . . . . . . 11 4.8. #9 Packet traceability . . . . . . . . . . . . . . . . . 12 4.9. #10 Technical maturity . . . . . . . . . . . . . . . . . 12 5. Data plane solution alternatives . . . . . . . . . . . . . . 12 5.1. DetNet Transport layer technologies . . . . . . . . . . . 13 5.1.1. Native IPv6 transport . . . . . . . . . . . . . . . . 13 5.1.2. Native IPv4 transport . . . . . . . . . . . . . . . . 16 5.1.3. Multiprotocol Label Switching (MPLS) . . . . . . . . 19 5.1.4. Bit Indexed Explicit Replication (BIER) . . . . . . . 23 - 5.1.5. BIER - Traffic Engineering (BIER-TE) . . . . . . . . 27 - 5.2. DetNet Service layer technologies . . . . . . . . . . . . 34 - 5.2.1. Generic Routing Encapsulation (GRE) . . . . . . . . . 34 - 5.2.2. MPLS-based Services for DetNet . . . . . . . . . . . 36 - 5.2.3. Pseudo Wire Emulation Edge-to-Edge (PWE3) . . . . . . 37 - 5.2.4. MPLS-Based Ethernet VPN (EVPN) . . . . . . . . . . . 41 - 5.2.5. Higher layer header fields . . . . . . . . . . . . . 43 - 6. Summary of data plane alternatives . . . . . . . . . . . . . 45 - 7. Security considerations . . . . . . . . . . . . . . . . . . . 47 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 47 - 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 48 - 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 - 10.1. Informative References . . . . . . . . . . . . . . . . . 48 - 10.2. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 57 + 5.2. DetNet Service layer technologies . . . . . . . . . . . . 27 + 5.2.1. Generic Routing Encapsulation (GRE) . . . . . . . . . 27 + 5.2.2. MPLS-based Services for DetNet . . . . . . . . . . . 29 + 5.2.3. Pseudo Wire Emulation Edge-to-Edge (PWE3) . . . . . . 30 + 5.2.4. MPLS-Based Ethernet VPN (EVPN) . . . . . . . . . . . 34 + 5.2.5. Higher layer header fields . . . . . . . . . . . . . 36 + 6. Summary of data plane alternatives . . . . . . . . . . . . . 38 + 7. Security considerations . . . . . . . . . . . . . . . . . . . 40 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 + 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 41 + 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 41 + 10.1. Informative References . . . . . . . . . . . . . . . . . 41 + 10.2. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Appendix A. Examples of combined DetNet Service and Transport - layers . . . . . . . . . . . . . . . . . . . . . . . 58 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 58 + layers . . . . . . . . . . . . . . . . . . . . . . . 50 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 50 1. Introduction Deterministic Networking (DetNet) [I-D.ietf-detnet-problem-statement] provides a capability to carry unicast or multicast data flows for real-time applications with extremely low data loss rates, timely delivery and bounded packet delay variation [I-D.finn-detnet-architecture]. The deterministic networking Quality of Service (QoS) is expressed as 1) the minimum and the maximum end- to-end latency from source (talker) to destination (listener), and 2) @@ -178,39 +177,39 @@ specific attributes, which are needed during forwarding and for service protection. DetNet enabled end systems originate and terminate the DetNet Service layer and are peers at the DetNet Service layer. DetNet relay and edge nodes also implement DetNet Service layer functions. The DetNet service layer is used to deliver traffic end to end across a DetNet domain. DetNet Transport Layer The DetNet transport layer is required on all DetNet nodes. All - DetNet nodes are end points and the transport layer. Non-DetNet + DetNet nodes are end points at the transport layer. Non-DetNet service aware transit nodes deliver traffic between DetNet nodes. The DetNet transport layer operates below and supports the DetNet Service layer and optionally provides congestion protection for DetNet flows. Distinguishing the function of these two DetNet data plane layers helps to explore and evaluate various combinations of the data plane solutions available. This separation of DetNet layers, while helpful, should not be considered as formal requirement. For example, some technologies may violate these strict layers and still be able to deliver a DetNet service. . . +-----------+ | Service | PW, RTP/(UDP), GRE +-----------+ - | Transport | (UDP)/IPv6, (UDP)/IPv4, MPLS LSPs, BIER, BIER-TE + | Transport | (UDP)/IPv6, (UDP)/IPv4, MPLS LSPs, BIER +-----------+ . . Figure 2: DetNet adaptation to data plane The two logical layers defined here aim to help to identify which data plane technology can be used for what purposes in the DetNet context. This layering is similar to the data plane concept of MPLS, where some part of the label stack is "Service" specific (e.g., PW @@ -400,38 +399,38 @@ encapsulated inside other protocols, for example, when transporting a layer-2 Ethernet frame over an IP transport network. In some cases a tunneling like encapsulation can be avoided by underlying transport protocol translation, for example, translating layer-2 Ethernet frame including addressing and flow identification into native IP traffic. Last it is possible that sources and destinations handle deterministic flows natively in layer-3. This criteria concerns what is the encapsulation method the solution alternative support: tunneling like encapsulation, protocol translation or native layer-3 transport. In addition to the encapsulation mechanism this criteria - is also concerned of the processing and specifically the encapsulate - header overhead. + is also concerned with the processing and specifically the + encapsulation header overhead. 4.2. #2 Flow identification The solution alternative has to provide means to identify specific - deterministic flows. The flow identification can, for example, be + deterministic flows. The flow identification can, for example, be an explicit field in the data plane encapsulation header or implicitly encoded into the addressing scheme of the used data plane protocol or their combination. This criteria concerns the availability and details of deterministic flow identification the data plane protocol alternative has. 4.3. #3 Packet sequencing and duplicate elimination The solution alternative has to provide means for end systems to number packets sequentially and transport that sequencing information - along with the sent packets. In addition to possible reordering + along with the sent packets. In addition to possible reordering of packets other important uses for sequencing are detecting duplicates and lost packets. In a case of intentional packet duplication a combination of flow identification and packet sequencing allows for detecting and eliminating duplicates at the destination (see Section 4.5 for more details). 4.4. #4 Explicit routes @@ -477,21 +476,21 @@ The IEEE 802.1CB (seamless redundancy) [IEEE8021CB] is an example of Ethernet-based solution that defines packet sequence numbering, flow duplication, flow merging, duplicate packet identification and elimination. The deterministic networking data plane solution alternative at layer-3 has to provide equivalent functionality. This criteria concerns the available mechanisms for packet replication and duplicate deletion the data plane protocol alternative has. 4.6. #6 Operations, Administration and Maintenance - The solution alternative should demonstrate an availability of + The solution alternative should demonstrate availability of appropriate standardized OAM tools that can be extended for deterministic networking purposes with a reasonable effort, when required. The OAM tools do not necessarily need to be specific to the data plane protocol as it could be the case, for example, with MPLS-based data planes. But any OAM-related implications or requirements on data plane hardware must be considered. The OAM includes but is not limited to tools listed in the requirements for overlay networks [I-D.ooamdt-rtgwg-ooam-requirement]. Specifically, the performance @@ -521,22 +520,22 @@ A critical DetNet service enabled by QoS (and perhaps CoS) is delivering zero congestion loss. There are different mechanisms that maybe used separately or in combination to deliver a zero congestion loss service. The key aspect of this objective is that DetNet packets are not discarded due to congestion at any point in a DetNet aware network. In the context of the data plane solution there should be means for flow identification, which then can be used to map a flow against - specific resources and treatment in a node enforcing the QoS. - Hereto, certain aspects of CoS and QoS may be provided by the + specific resources and treatment in a node enforcing the QoS. For + DetNet, certain aspects of CoS and QoS may be provided by the underlying sub-net technology, e.g., actual queuing or IEEE 802.3x priority flow control (PFC). 4.8. #9 Packet traceability For the network management and specifically for tracing implementation or network configuration errors any means to find out whether a packet is a replica, which node performed replication, and which path was intended for the replica, can be very useful. This criteria concerns the availability of solutions for tracing packets @@ -747,21 +746,21 @@ 5.1.2. Native IPv4 transport 5.1.2.1. Solution description IPv4 [RFC0791] is in principle the same as IPv6, except that it has a smaller address space. However, IPv6 was designed around the fact that extension headers are an integral part of the protocol and operation from the beginning, although the practice may some times prove differently [RFC7872]. IPv4 does support header options, but - these have historically not been supported on in hardware-based + these have historically not been supported in hardware-based forwarding so are generally blocked or handled at a much slower rate. In either case, the use of IP header options is generally avoided. In the context of deterministic networking data plane solutions the major difference between IPv4 and IPv6 seems to be the practical support for header extensibility. Anything below and above the IP header independent of the version is practically the same. 5.1.2.2. Analysis and Discussion #1 Encapsulation and overhead (M) @@ -897,23 +896,23 @@ ~~~~~~~~~~~ denotes DetNet Service <-> DetNet Transport layer boundary Figure 7: MPLS-based Services MPLS can be controlled in a number of ways including via a control plane, via the management plane, or via centralized controller (SDN) based approaches. MPLS also provides standard control plane reference points. Additional information on MPLS architecture and control can be found in [RFC5921]. A summary of MPLS control plane related functions can be found in [RFC6373]. The remainder of this - section will focus [RFC6373]. The remainder of this section will - focus on the MPLS transport data plane, additional information on the - MPLS service data plane can be found below in Section 5.2.2. + section will focus on the MPLS transport data plane, additional + information on the MPLS service data plane can be found below in + Section 5.2.2. 5.1.3.1. Solution description The following draws heavily from [RFC5960]. Encapsulation and forwarding of packets traversing MPLS LSPs follows standard MPLS packet encapsulation and forwarding as defined in [RFC3031], [RFC3032], [RFC5331], and [RFC5332]. Data plane Quality of Service capabilities are included in the MPLS @@ -973,26 +972,26 @@ 5.1.3.2. Analysis and Discussion #1 Encapsulation and overhead (M) There are two perspectives to consider when looking at encapsulation. The first is encapsulation to support services. These considerations are part of the DetNet service layer and are covered below, see Sections 5.2.3 and 5.2.4. - The second perspective relates to encapsulation, if any, is needed - to transport packets across network. In this case, the MPLS label - stack, [RFC3032] is used to identify flows across a network. MPLS - labels are compact and highly flexible. They can be stacked to - support client adaptation, protection, network layering, source - routing, etc. + The second perspective relates to encapsulation, if any, which is + needed to transport packets across network. In this case, the + MPLS label stack, [RFC3032] is used to identify flows across a + network. MPLS labels are compact and highly flexible. They can + be stacked to support client adaptation, protection, network + layering, source routing, etc. The number of DetNet Transport layer specific labels is flexible and support a wide range of applicable functions and MPLS domain characteristics (e.g., TE-tunnels, Hierarchical-LSPs, etc.). #2 Flow identification (M) MPLS label stacks provide highly flexible ways to identify flows. Basically, they enable the complete separation of traffic classification from traffic treatment and thereby enable arbitrary combinations of both. @@ -1007,22 +1006,22 @@ MPLS supports explicit routes based on how LSPs are established, e.g., via TE explicit routes [RFC3209]. Additional, but not required, capabilities are being defined as part of Segment Routing (SR) [I-D.ietf-spring-segment-routing]. #5 Flow duplication and merging (M/W) MPLS as DetNet Transport layer supports the replication via point- to-multipoint LSPs. At the MPLS LSP level, there are mechanisms - defined to provide 1+1 protection, which could help realizing the - flow merging function. The current definitions [RFC6378] and + defined to provide 1+1 protection, which could help in realizing + the flow merging function. The current definitions [RFC6378] and [RFC7271] use OAM mechanisms to support and coordinate protection switching and packet loss is possible during a switch. While such this level of protection may be sufficient for many DetNet applications, when truly hitless (i.e., zero loss) switching is required, additional mechanisms will be needed. It is expected that these additional mechanisms will be defined at a DetNet layer. #6 Operations, Administration and Maintenance (M) @@ -1068,39 +1067,27 @@ 5.1.4. Bit Indexed Explicit Replication (BIER) Bit Indexed Explicit Replication [I-D.ietf-bier-architecture] (BIER) is a network plane replication technique that was initially intended as a new method for multicast distribution. In a nutshell, a BIER header includes a bitmap that explicitly signals the destinations that are intended for a particular packet, which means that 1) the source is aware of the individual destinations and 2) the BIER control plane is a simple extension of the unicast routing as opposed to a dedicated multicast data plane, which represents a considerable - reduction in OPEX. For this reason, the technology faces a lot of - traction from Service Providers. Section 5.1.4 discusses the - applicability of BIER for replication in the DetNet. - - The simplicity of the BIER technology makes it very versatile as a - network plane signaling protocol. Already, a new Traffic Engineering - variation is emerging that uses bits to signal segments along a TE - path. While the more classical BIER is mainly a multicast technology - that typically leverages a unicast distributed control plane through - IGP extensions, BIER-TE is mainly a unicast technology that leverages - a central computation to setup path, compute segments and install the - mapping in the intermediate nodes. Section 5.1.5 discusses the - applicability of BIER-TE for replication, traceability and OAM - operations in DetNet. - - Bit-Indexed Explicit Replication (BIER) layer may be considered to be - included into Deterministic Networking data plane solution. - Encapsulation of a BIER packet in MPLS network presented in Figure 8 + reduction in OPEX. For this reason, the technology is getting a lot + of traction from Service Providers. In the context of DetNet, BIER + may be applicable for implementing packet replication, as described + in Section 5.1.4. + The encapsulation of a BIER packet in an MPLS network is shown in + Figure 8 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label Stack Element | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label Stack Element | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | BIER-MPLS label | |1| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 1 0 1| Ver | Len | Entropy | @@ -1111,35 +1098,35 @@ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ BitString (last 32 bits) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |OAM| Reserved | Proto | BFIR-id | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8: BIER packet in MPLS encapsulation 5.1.4.1. Solution description - The DetNet may be presented in BIER as distinctive payload type with - its own Proto(col) ID. Then it is likely that DetNet will have the - header that would identify: + A distinctive BIER payload type (with its own Proto(col) ID) would be + created for DetNet, providing a header that would identify: o Version; o Sequence Number; o Timestamp; o Payload type, e.g. data vs. OAM. - DetNet node, collocated with BFIR, may use multiple BIER sub-domains - to create replicated flows. Downstream DetNet nodes, collocated with - BFER, would terminate redundant flows based on Sequence Number and/or - Timestamp information. Such DetNet may be BFER in one BIER sub- - domain and BFIR in another. Thus DetNet flow would traverse several - BIER sub-domains. + A DetNet node, collocated with a BFIR (Bit-Forwarding Ingress Router) + may use multiple BIER sub-domains to create replicated flows. + Downstream DetNet nodes, collocated with BFER (Bit-Forwarding Ingress + Router) would terminate redundant flows based on Sequence Number and/ + or Timestamp information. Thus a DetNet node may be collocated with + a BFER in one BIER sub- domain and with a BFIR in another, and a + DetNet flow could traverse several BIER sub-domains. +-----+ | A | +-----+ / \ . . / . . \ / . . . @@ -1163,79 +1150,80 @@ . . . / \ . . . . . . . \ . . / +-----+ +-----+ | G | | H | +-----+ +-----+ Figure 9: DetNet in BIER domain - Consider DetNet flow that must traverse BIER enabled domain from A to - G and H. DetNet may use three BIER subdomains: + Consider a DetNet flow that must traverse BIER enabled domain from A + to G and H. DetNet may use three BIER subdomains: o A-B-D-E-G (dash-dot): A is BFIR, E and G are BFERs, o A-C-E-F-H (dash-double-dot): A is BFIR, E and H are BFERs, o E-G-H (dotted): E is BFIR, G and H are BFERs. DetNet node A sends DetNet into red and purple BIER sub-domains. DetNet node E receives DetNet packet and sends into green sub-domain - while terminating duplicates and those that deemed too-late. + while terminating duplicates and those that are deemed "too-late". DetNet nodes G and H receive DetNet flows, terminate duplicates and those that are too-late. 5.1.4.2. Analysis and Discussion #1 Encapsulation and overhead (M) - BIER over MPLS network encapsulation (will refer as "BIER over - MPLS" further for short), Figure 8, is being defined [I-D. ietf- - bier-mpls-encapsulation] within the BIER working group. + BIER over MPLS network encapsulation ("BIER over MPLS"), Figure 8, + is being defined [I-D.ietf-bier-mpls-encapsulation] within the + BIER working group. #2 Flow identification (M) Flow identification and separation can be achieved through use of BIER domains and/or Entropy value in the BIER over MPLS, Figure 8. #4 Explicit routes (M) Explicit routes may be used as underlay for BIER domain. BIER underlay may be calculated using PCE and instantiated using any southbound mechanism. #5 Flow duplication and merging (M/W) - Packet replication, as indicated by its name, is core function of - the Bit-Indexed Explicit Replication. Elimination of the + Packet replication, as indicated by its name, is a core function + of the Bit-Indexed Explicit Replication. Elimination of the duplicates and/or too-late packets cannot be done within BIER sub- domain but may be done at DetNet overlay at the edge of the BIER sub-domain. [Editor's note: how about the flow merging?] #6 Operations, Administration and Maintenance (M/W) BIER over MPLS guarantees that OAM is fate-sharing, i.e. in-band with a data flow being monitored or measured. Additionally, BIER over MPLS enables passive performance measurement, e.g. with the - marking method [I-D.mirsky-bier-pmmm-oam]. Some OAM protocols, - e.g. can be applied and used in BIER over MPLS as demonstrated - [I-D.ooamdt-rtgwg-oam-gap-analysis], while new protocols being + marking method [I-D.mirsky-bier-pmmm-oam]. Existing OAM protocols + can be applied and used in BIER over MPLS as demonstrated in + [I-D.ooamdt-rtgwg-oam-gap-analysis], while new protocols are being worked on, e.g. ping/traceroute [I-D.kumarzheng-bier-ping] or Path MTU Discovery [I-D.mirsky-bier-path-mtu-discovery]. #8 Class and quality of service capabilities (M/W) + Class of Service can be inherited from the underlay of the particular BIER sub-domain. Quality of Service, i.e. scheduling - and bandwidth reservations can be used among other constrains in - calculating explicit path for the BIER sub-domain's underlay. + and bandwidth reservations can be used among other constraints in + calculating explicit paths for the BIER sub-domain's underlay. #9 Packet traceability (W) Ability to do passive performance measurement by using OAM field of the BIER over MPLS, Figure 8, is unmatched and significantly simplifies truly passive tracing of selected flows and packets within them. #10 Technical maturity (W) @@ -1244,367 +1232,68 @@ 5.1.4.3. Summary BIER over MPLS supports a significant portion of the identified DetNet data plane requirements, including controlled packet replication, traffic engineering, while some requirements, e.g. duplicate and too-late packet elimination may be realized as function of the DetNet overlay. BIER over MPLS is a viable candidate as the DetNet Transport layer in MPLS networks. -5.1.5. BIER - Traffic Engineering (BIER-TE) - - An alternate use of Bit-Indexed Explicit Replication (BIER) uses bits - in the BitString to represent adjacencies as opposed to destinations, - as discussed in BIER Traffic Engineering (TE) - [I-D.eckert-bier-te-arch]. - - The proposed function of BIER-TE in the DetNet data plane is to - control the process of replication and elimination, as opposed to the - identification of the flows or and the sequencing of packets within a - flow. - - At the path ingress, BIER-TE identifies the adjacencies that are - activated for this packet (under the rule of the controller). At the - egress, BIER-TE is used to identify the adjacencies where - transmission failed. This information is passed to the controller, - which in turn can modify the active adjacencies for the next packets. - - The value is that the replication can be controlled and monitored in - a loop that may involve an external controller, with the granularity - of a packet and an adjacency . - -5.1.5.1. Solution description - - BIER-TE enables to activate the replication and elimination functions - in a manner that is abstract to the data plane forwarding - information. An adjacency, which is represented by a bit in the BIER - header, can correspond in the data plane to an Ethernet hop, a Label - Switched Path, or it can correspond to an IPv6 loose or strict source - routed path. - - In a nutshell, BIER-TE is used as follows: - - o A controller computes a complex path, sometimes called a track, - which takes the general form of a ladder. The steps and the side - rails between them are the adjacencies that can be activated on - demand on a per-packet basis using bits in the BIER header. - - ===> (A) ====> (C) ==== - // ^ | ^ | \\ - ingress (I) | | | | (E) egress - \\ | v | v // - ===> (B) ====> (D) ==== - - Figure 10: Ladder Shape with replication and elimination Points - - o The controller assigns a BIER domain, and inside that domain, - assigns bits to the adjacencies. The controller assigns each bit - to a replication node that sends towards the adjacency, for - instance the ingress router into a segment that will insert a - routing header in the packet. A single bit may be used for a step - in the ladder, indicating the other end of the step in both - directions. - - ===> (A) ====> (C) ==== - // 1 ^ | 4 ^ | 7 \\ - ingress (I) |2| |6| (E) egress - \\ 3 | v 5 | v 8 // - ===> (B) ====> (D) ==== - - Figure 11: Assigning Bits - - o The controller activates the replication by deciding the setting - of the bits associated with the adjacencies. This decision can be - modified at any time, but takes the latency of a controller round - trip to effectively take place. Below is an example that uses - replication and elimination to protect the A->C adjacency. - - +-------+-----------+-------+---------------------+ - | Bit # | Adjacency | Owner | Example Bit Setting | - +-------+-----------+-------+---------------------+ - | 1 | I->A | I | 1 | - | 2 | A->B | A | 1 | - | | B->A | B | | - | 3 | I->C | I | 0 | - | 4 | A->C | A | 1 | - | 5 | B->D | B | 1 | - | 6 | C->D | C | 1 | - | | D->C | D | | - | 7 | C->E | C | 1 | - | 8 | D->E | D | 0 | - +-------+-----------+-------+---------------------+ - - replication and elimination Protecting A->C - - Table 1: Controlling Replication - - o The BIER header with the controlling BitString is injected in the - packet by the ingress node of the deterministic path. That node - may act as a replication point, in which case it may issue - multiple copies of the packet - - ====> Repl ===> Elim ==== - // | ^ \\ - ingress | | egress - v | - Fwd ====> Fwd - - Figure 12: Enabled Adjacencies - - o For each of its bits that is set in the BIER header, the owner - replication point resets the bit and transmits towards the - associated adjacency; to achieve this, the replication point - copies the packet and inserts the relevant data plane information, - such as a source route header, towards the adjacency that - corresponds to the bit - +-----------+----------------+ - | Adjacency | BIER BitString | - +-----------+----------------+ - | I->A | 01011110 | - | A->B | 00011110 | - | B->D | 00010110 | - | D->C | 00010010 | - | A->C | 01001110 | - +-----------+----------------+ - - BitString in BIER Header as Packet Progresses - - Table 2: BIER-TE in Action - - o Adversely, an elimination node on the way strips the data plane - information and performs a bitwise AND on the BitStrings from the - various copies of the packet that it has received, before it - forwards the packet with the resulting BitString. - - +-----------+----------------+ - | Operation | BIER BitString | - +-----------+----------------+ - | D->C | 00010010 | - | A->C | 01001110 | - | | -------- | - | AND in C | 00000010 | - | | | - | C->E | 00000000 | - +-----------+----------------+ - - BitString Processing at Elimination Point C - - Table 3: BIER-TE in Action (cont.) - - o In this example, all the transmissions succeeded and the BitString - at arrival has all the bits reset - note that the egress may be an - Elimination Point in which case this is evaluated after this node - has performed its AND operation on the received BitStrings). - - +-------------------+-----------------------+ - | Failing Adjacency | Egress BIER BitString | - +-------------------+-----------------------+ - | I->A | Frame Lost | - | I->B | Not Tried | - | A->C | 00010000 | - | A->B | 01001100 | - | B->D | 01001100 | - | D->C | 01001100 | - | C->E | Frame Lost | - | D->E | Not Tried | - +-------------------+-----------------------+ - - BitString indicating failures - - Table 4: BIER-TE in Action (cont.) - - o But if a transmission failed along the way, one (or more) bit is - never cleared. Table 4 provides the possible outcomes of a - transmission. If the frame is lost, then it is probably due to a - failure in either I->A or C->E, and the controller should enable - I->B and D->E to find out. A BitString of 00010000 indicates - unequivocally a transmission error on the A->C adjacency, and a - BitString of 01001100 indicates a loss in either A->B, B->D or - D->C; enabling D->E on the next packets may provide more - information to sort things out. - - In more details: - - The BIER header is of variable size, and a DetNet network of a - limited size can use a model with 64 bits if 64 adjacencies are - enough, whereas a larger deployment may be able to signal up to 256 - adjacencies for use in very complex paths. Figure 8 illustrates a - BIER header as encapsulated within MPLS. The format of this header - is common to BIER and BIER-TE. - - For the DetNet data plane, a replication point is an ingress point - for more than one adjacency, and an elimination point is an egress - point for more than one adjacency. - - A pre-populated state in a replication node indicates which bits are - served by this node and to which adjacency each of these bits - corresponds. With DetNet, the state is typically installed by a - controller entity such as a PCE. The way the adjacency is signaled - in the packet is fully abstracted in the bit representation and must - be provisioned to the replication nodes and maintained as a local - state, together with the timing or shaping information for the - associated flow. - - The DetNet data plane uses BIER-TE to control which adjacencies are - used for a given packet. This is signaled from the path ingress, - which sets the appropriate bits in the BIER BitString to indicate - which replication must happen. - - The replication point clears the bit associated to the adjacency - where the replica is placed, and the elimination points perform a - logical AND of the BitStrings of the copies that it gets before - forwarding. - - As is apparent in the examples above, clearing the bits enables to - trace a packet to the replication points that made any particular - copy. BIER-TE also enables to detect the failing adjacencies or - sequences of adjacencies along a path and to activate additional - replications to counter balance the failures. - - Finally, using the same BIER-TE bit for both directions of the steps - of the ladder enables to avoid replication in both directions along - the crossing adjacencies. At the time of sending along the step of - the ladder, the bit may have been already reset by performing the AND - operation with the copy from the other side, in which case the - transmission is not needed and does not occur (since the control bit - is now off). - -5.1.5.2. Analysis and Discussion - - #1 Encapsulation and overhead (W/M) - - The size of the BIER header depends on the number of segments in - the particular path. It is very concise considering the amount of - information that is carried (control of replication, traceability, - and measurement of the reliability of the segments). - - #2 Flow identification (N) - - Some fields in the BIER header could be used to identify the flows - but they are not the primary purpose, so it's probably not a good - idea. - - #4 Explicit routes (N) - - A separate procedure must be used to set up the paths and allocate - the bits for the adjacencies. The bits should be distributed as a - form of tag by the route setup protocol. This procedure requires - more work and is separate from the data plane method that is - described here. - - #5 Flow duplication and merging M/W) - - The bitmap expresses in a very concise fashion which replication - and merging (and elimination) should take place for a given - packet. It also enables to control that process on a per packet - basis, depending on the loss that it enables to measure. The net - result is that a complex path may be installed with all the - possibilities and that the decision of which possibilities are - used is controlled in the data plane. - - #6 Operations, Administration and Maintenance (W) - - The setting of the bits at arrival enables to determine which - adjacencies worked and which did not, enabling a dynamic control - of the replication and elimination process. This is a form of OAM - that is in-band with the data stream as opposed to leveraging - separate packets, which is a more accurate information on the - reliability of the link for the user. - - #8 Class and quality of service capabilities (N) - - BIER-TE does not signal that explicitly. - - #9 Packet traceability (W) - - This is a strong point of the solution. The solution enables to - determine which is the current segment that a given packet is - expected to traverse, which node performed the replication and - which should perform the elimination if any - - #10 Technical maturity (W) - - Some components of the technology are more mature, e.g. segment - routing and BIER. Yet, the overall solution has never been - deployed as is not fully defined. It should be noted that the - definition of the BIER-TE solution is outside the scope of the - DetNet WG charter. - -5.1.5.3. Summary - - BIER-TE occupies a particular position in the DetNet data plane. In - the one hand it is optional, and only useful if replication and - elimination is taking place. In the other hand, it has unique - capabilities to: - - o control which replication take place on a per packet basis, so - that replication points can be configured but not actually - utilized - o trace the replication activity and determine which node replicated - a particular packet - o measure the quality of transmission of the actual data packet - along the replication segments and use that in a control loop to - adapt the setting of the bits and maintain the reliability. - - However, as noted earlier, BIER-TE is not yet fully specified and the - required specification work is outside the scope of the current - DetNet WG charter. - 5.2. DetNet Service layer technologies 5.2.1. Generic Routing Encapsulation (GRE) 5.2.1.1. Solution description Generic Routing Encapsulation (GRE) [RFC2784] provides an encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol. The encapsulation of a GRE packet - can be found in Figure 13. + can be found in Figure 10. +-------------------------------+ | Delivery Header | +-------------------------------+ | GRE Header | +-------------------------------+ | Payload packet | +-------------------------------+ - Figure 13: Encapsulation of a GRE packet + Figure 10: Encapsulation of a GRE packet Based on RFC2784, [RFC2890] further includes sequencing number and Key in optional fields of the GRE header, which may help to transport DetNet traffic flows over IP networks. The format of a GRE header is - presented in Figure 14. + presented in Figure 11. 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C| |K|S| Reserved0 | Ver | Protocol Type | +-----------------------------------------------------------------+ | Checksum (optional) | Reserved1 (Optional) | +-----------------------------------------------------------------+ | Key (optional) | +-----------------------------------------------------------------+ | Sequence Number (optional) | +-----------------------------------------------------------------+ - Figure 14: Format of a GRE header + Figure 11: Format of a GRE header 5.2.1.2. Analysis and Discussion #1 Encapsulation and overhead (M) GRE can provide encapsulation at the service layer over the transport layer. A new protocol type for DetNet traffic should be allocated as an "Ether Type" in [RFC3232] and in IANA Ethernet Numbers [5]. The fixed header of a GRE packet is 4 octets while - the maximum header is 16 octets with optional fields in Figure 14. + the maximum header is 16 octets with optional fields in Figure 11. #2 Flow identification (W) There is no flow identification field in GRE header. However, it can rely on the flow identification mechanism applied in the delivery protocols, such as flow identification stated in IP Sections 5.1.1 and 5.1.2 when the delivery protocols are IPv6 and IPv4 respectively. Alternatively, the Key field can also be extended to carry the flow identification. The size of Key field is 4 octets. @@ -1651,52 +1340,52 @@ network layer protocols over another network layer, which can naturally serve as the service layer protocol for DetNet. Currently, it supports a portion of the Detnet service layer criteria, and still some are not fully supported but can be incrementally added or supported by delivery protocols at as the transport layer. In general, GRE can be a choice as the DetNet service layer and can work with IPv6 and IPv4 as the DetNet Transport layer. 5.2.2. MPLS-based Services for DetNet - MPLS based technologies supports both the DetNet Service and DetNet + MPLS based technologies support both the DetNet Service and DetNet Transport layers. This, as well as a general overview of MPLS, is - covered above in Section 5.1.3. These sections focus on the DetNet - Service Layer it provides client service adaption, via Pseudowires - Section 5.2.3 and via native and other label-like mechanisms such as - EPVN in Section 5.2.4. A representation of these options was - previously discussed and is shown in Figure 7. + covered above in Section 5.1.3. Following sections focus on the + DetNet Service Layer which provides client service adaption, via + Pseudowires Section 5.2.3 and via native and other label-like + mechanisms such as EPVN in Section 5.2.4. A representation of these + options was previously discussed and is shown in Figure 7. The following text is adapted from [RFC5921]: The MPLS native service adaptation functions interface the client layer network service to MPLS. For Pseudowires, these adaptation functions are the payload encapsulation described in Section 4.4 of [RFC3985] and Section 6 of [RFC5659]. For network layer client services, the adaptation function uses the MPLS encapsulation format as defined in [RFC3032]. The purpose of this encapsulation is to abstract the data plane of the client layer network from the MPLS data plane, thus contributing to the independent operation of the MPLS network. MPLS may itself be a client of an underlying server layer. MPLS - can thus also bounded by a set of adaptation functions to this + can thus also be bounded by a set of adaptation functions to this server layer network, which may itself be MPLS. These adaptation functions provide encapsulation of the MPLS frames and for the transparent transport of those frames over the server layer network. - While MPLS service can provided on and true end-system to end- - system basis, it's more likely that DetNet service will be - provided over Pseudowires as described in Section 5.2.3 or via an - EPVN-based service described in Section 5.2.4 . + While MPLS service can provided on a true end-system to end-system + basis, it's more likely that DetNet service will be provided over + Pseudowires as described in Section 5.2.3 or via an EPVN-based + service described in Section 5.2.4 . MPLS labels in the label stack may be used to identify transport paths, see Section 5.1.3, or as service identifiers. Typically a single label is used for service identification. Packet sequencing mechanisms are added in client-related adaptation processing, see Sections 5.2.3 and 5.2.4. The MPLS client inherits its Quality of Service (QoS) from the MPLS transport layer, which in turn inherits its QoS from the @@ -1707,42 +1396,42 @@ 5.2.3. Pseudo Wire Emulation Edge-to-Edge (PWE3) 5.2.3.1. Solution description Pseudo Wire Emulation Edge-to-Edge (PWE3) [RFC3985] or simply PseudoWires (PW) provide means of emulating the essential attributes and behaviour of a telecommunications service over a packet switched network (PSN) using IP or MPLS transport. In addition to traditional telecommunications services such as T1 line or Frame Relay, PWs also provide transport for Ethernet service [RFC4448] and for generic - packet service [RFC6658]. Figure 15 illustrate the reference PWE3 + packet service [RFC6658]. Figure 12 illustrate the reference PWE3 stack model. +----------------+ +----------------+ |Emulated Service| |Emulated Service| |(e.g., Eth, ...)|<= Emulated Service =>|(e.g., Eth, ...)| +----------------+ +----------------+ | Payload | | Payload | CW, | Encapsulation |<=== Pseudo Wire ====>| Encapsulation | Timing, | | | | Seq., .. +----------------+ +----------------+ |PW Demultiplexer| |PW Demultiplexer| | PSN Tunnel, |<==== PSN Tunnel ====>| PSN Tunnel, | MPLS. | PSN & Physical | | PSN & Physical | L2TP, | Layers | | Layers | IP, .. +-------+--------+ ___________ +---------+------+ | / \ | +============/ PSN \==============+ \ / \___________/ - Figure 15: PWE3 protocol stack reference model + Figure 12: PWE3 protocol stack reference model PWs appear as a good data plane solution alternative for a number of reasons. PWs are a proven and deployed technology with a rich OAM control plane [RFC4447], and enjoy the toolbox developed for MPLS networks in a case of MPLS-based PSN. Furthermore, PWs may have an optional Control Word (CW) as part of the payload encapsulation between the PSN and the emulated service that is, for example, capable of frame sequencing and duplicate detection. The encapsulation layer may also provide timing using RTP as described in Sections 5.2.2, 5.4.1 and 5.4.2 of [RFC3985] and utilized by @@ -1771,23 +1460,24 @@ Port indicates tunneled MPLS packet and the UDP Source Port is an entropy value that is generated by the encapsulator to uniquely identify a flow. MPLS-in-UDP encapsulation can be applied to enable UDP-based ECMP (Equal-Cost Multipath) or Link Aggregation. All these solutions can be secured with IPsec [RFC4303]. 5.2.3.2. Analysis and Discussion #1 Encapsulation and overhead (M) - PWs offer encapsulation services practically for any types of - payloads over any PSN. New PW types need a code point allocation - [RFC4446] and in some cases an emulated service specific document. + PWs offer encapsulation services practically for practically any + type of payload over any PSN. New PW types need a code point + allocation [RFC4446] and in some cases an emulated service + specific document. Specifically in the case of the MPLS PSN the PW encapsulation overhead is minimal. Typically minimum two labels and a CW is needed, which totals to 12 octets. PW type specific handling might, however, allow optimizations on the emulated service in the provider edge (PE) device's native service processing (NSP) / forwarder function. These optimizations could be used, for example, to reduce header overhead. Ethernet PWs already have rather low overhead [RFC4448]. Without a CW and VLAN tags the Ethernet header gets reduced to 14 octets (minimum Ethernet header @@ -1798,21 +1488,21 @@ or 40 (IPv6) bytes overhead to the PW over MPLS overhead; furthermore, the GRE, L2TPv3, or UDP header has to be taken into account if any of these further encapsulations is used. #2 Flow identification (M) PWs provide multiple layers of flow identification, especially in the case of the MPLS PSN. The PWs are typically prepended with an endpoint specific PW label that can be used to identify a specific PW per endpoint. Furthermore, the MPLS PSN also uses one or more - labels to transport packets over a specific label switched paths + labels to transport packets over specific label switched paths (that then would carry PWs). So, a DetNet flow can be identified in this example by the service and transport layer labels. IP (and other) PSNs may need other mechanisms, such as, UDP port numbers, upper layer protocol header (like RTP) or some IP extension header to provide required flow identification. #3 Packet sequencing and duplicate elimination (M) As mentioned earlier PWs may contain an optional CW that is able to provide sequencing services. The size of the sequence number @@ -1820,55 +1510,55 @@ used link and DetNet flow speed be too little. The PW duplicate detection mechanism is already conceptually specified [RFC3985] but no emulated service makes use of it currently. #5 Flow duplication and merging (W) PWs could use a (extended) version of existing transport layer provided protection mechanisms (e.g., hitless 1+1 protection) for both flow duplication and flow merging. The service layer has to provide the functionality to map DetNet flows into appropriate - transport leyer connection, though. + transport layer connection. #6 Operations, Administration and Maintenance (M/W) PWs have rich control plane for OAM and in a case of the MPLS PSN enjoy the full control plane toolbox developed for MPLS network - OAM likewise IP PSN have the full toolbox of IP network OAM tools. + OAM likewise IP PSN has the full toolbox of IP network OAM tools. There could be, however, need for deterministic networking specific extensions for the mentioned control planes. #8 Class and quality of service capabilities (M/W) In a case of IP PSN the 6-bit differentiated services code point (DSCP) field can be used for indicating the class of service [RFC2474] and 2-bit field reserved for the explicit congestion notification (ECN) [RFC3168]. Similarly, in a case of MPLS PSN, there are 3-bit traffic class field (TC) [RFC5462] in the label reserved for for both Explicitly TC-encoded-PSC LSPs (E-LSP) [RFC3270] and ECN [RFC5129]. Due to the limited number of bits in - the TC field, their use for QoS and ECN functions restricted and - intended to be flexible. Although the QoS/CoS mechanism is + the TC field, their use for QoS and ECN functions is restricted + and intended to be flexible. Although the QoS/CoS mechanism is already in place some clarifications may be required in the context of deterministic networking flows, for example, if some specific mapping between bit fields have to be done. When PWs are used over MPLS, MPLS LSPs can be used to provide both CoS (E-LSPs and L-LSPs) and QoS (dedicated TE LSPS). #10 Technical maturity (M) PWs, IP and MPLS are proven technologies with wide variety of deployments and years of operational experience. Furthermore, the estimated work for missing functionality (packet replication and elimination) does not appear to be extensive, since the existing - protection mechanism already get close to what is needed from the + protection mechanism already gets close to what is needed from the deterministic networking data plane solution. 5.2.3.3. Summary PseudoWires appear to be a strong candidate as the deterministic networking data plane solution alternative for the DetNet Service layer. The strong points are the technical maturity and the extensive control plane for OAM. This holds specifically for MPLS- based PSN. @@ -1988,24 +1678,24 @@ identification. 5.2.5.2. RTP 5.2.5.2.1. Solution Description Real-time Transport Protocol (RTP) [RFC3550] is often used to deliver time critical traffic in IP networks. RTP is typically carried on top of UDP/IP. However, as noted earlier in Section 5.2.3 PseudoWires also have a well-defined way of embedding and - transposting RTP header as part of its payload encapsulation headers/ - sub-layer. RTP is also augmented by its own control protocol RTCP, - which monitors of the data delivery and provides minimal control and - identification functionality. RTCP packets do not carry "media + transposting RTP headers as part of its payload encapsulation + headers/sub-layer. RTP is also augmented by its own control protocol + RTCP, which monitors the data delivery and provides minimal control + and identification functionality. RTCP packets do not carry "media payload". Although both RTP and RTCP are typically used with UDP/IP transport they are designed to be independent of the underlying transport and network layers. The RTP header includes a 2-byte sequence number, which can be used to detect and eliminate duplicate packets if DetNet service protection is used. The sequence number is conveyed by bits 16 through 31 of the RTP header. In addition to the sequence number the RTP header has also timestamp field (bits 32 through 63) that can be useful for time synchronization purposes. Furthermore, the RTP @@ -2022,21 +1712,21 @@ when RTP is transported over IP. Although RTCP packets do not contribute to the media payload transport they still consume overall network capacity, since all participants to an RTP session including sourcess and multicast session destinations are expected to send RTCP reports. #2 Flow identification (M) The RTP header contains a synchronization source (SSRC) identifier. The intent is that no two synchronization sources - within the same RTP session has the same SSRC identifier. + within the same RTP session have the same SSRC identifier. #3 Packet sequencing and duplicate elimination (M) The RTP header contains a 16 bit sequence number. The sequence number can be also used to detect duplicate packets. #5 Flow duplication and merging (M/W) RTP has precedence of being used for hitless protection switching [ST20227], which essentially is equivalent to DetNet service @@ -2086,70 +1776,69 @@ | #2 | Flow identification | | #3 | Packet sequencing and duplicate elimination | | #4 | Explicit routes | | #5 | Flow duplication and merging | | #6 | Operations, Administration and Maintenance | | #8 | Class and quality of service capabilities | | #9 | Packet traceability | | #10 | Technical maturity | +--------+---------------------------------------------+ - Table 5: Evaluation criteria (#7 obsoleted) + Table 1: Evaluation criteria There is no single technology that could meet all the criteria on its own. Distinguishing the DetNet Service and the DetNet Transport, as explained in (Section 3), allows a number of combinations, which can meet most of the criteria. There is no room here to evaluate all possible combinations. Therefore, only some combinations are highlighted here, which are selected based on the number of criteria that are met and the maturity of the technology (#10). The following table summarizes the evaluation of the data plane options that can be used for the DetNet Transport Layer against the evaluation criteria. Each value in the table is from the corresponding section. Applicability per Transport Alternative - +----------+-----+----+----+-----+-----+-----+----+-----+ + +----------+----+----+----+-----+-----+-----+----+-----+ | Solution | #1 | #2 | #4 | #5 | #6 | #8 | #9 | #10 | - +----------+-----+----+----+-----+-----+-----+----+-----+ + +----------+----+----+----+-----+-----+-----+----+-----+ | IPv6 | M | W | W | W | M | W | W | M/W | | IPv4 | M | W | W | W/N | M | M/W | W | M/W | | MPLS | M | M | M | M/W | M | M/W | M | M | | BIER | M | M | M | M/W | M/W | M/W | M | W | - | BIER-TE | W/M | N | N | M/W | W | N | W | W | - +----------+-----+----+----+-----+-----+-----+----+-----+ + +----------+----+----+----+-----+-----+-----+----+-----+ Summarizing Transport capabilities - Table 6: DetNet Transport Layer + Table 2: DetNet Transport Layer The following table summarizes the evaluation of the data plane options that can be used for the DetNet Service Layer against the criteria evaluation criteria. Each value in the table is from the corresponding section. Applicability per Service Alternative +----------+----+----+-----+-----+-----+-----+-----+ | Solution | #1 | #2 | #3 | #5 | #6 | #8 | #10 | +----------+----+----+-----+-----+-----+-----+-----+ | GRE | M | W | M/W | W/N | M | W | M | | PWE3 | M | M | M | W | M/W | M/W | M | | EVPN | M | W | M | M/W | M/W | M/W | M | | RTP | M | M | M | M/W | M | M/W | M | +----------+----+----+-----+-----+-----+-----+-----+ Summarizing Service capabilities - Table 7: DetNet Service Layer + Table 3: DetNet Service Layer PseudoWire (Section 5.2.3) is a technology that is mature and meets most of the criteria for the DetNet Service layer as shown in the table above. From upper layer protocols PWs or RTP can be a candidate for non-MPLS PSNs. The identified work for PWs is to figure out how to implement duplicate detection for these protocols (e.g., based on [RFC3985]). In a case of RTP there is precedence of implementing packet duplication and duplicate elimination [ST20227][RFC7198]. @@ -2191,115 +1880,101 @@ The DetNet chairs serving during the DetNet Data Plane Design Team: Lou Berger Pat Thaler 10. References 10.1. Informative References - [I-D.eckert-bier-te-arch] - Eckert, T., Cauchie, G., Braun, W., and M. Menth, "Traffic - Engineering for Bit Index Explicit Replication BIER-TE", - draft-eckert-bier-te-arch-04 (work in progress), July - 2016. - [I-D.finn-detnet-architecture] - Finn, N., Thubert, P., and M. Teener, "Deterministic - Networking Architecture", draft-finn-detnet- - architecture-07 (work in progress), July 2016. + Finn, N. and P. Thubert, "Deterministic Networking + Architecture", draft-finn-detnet-architecture-08 (work in + progress), August 2016. [I-D.ietf-6man-rfc2460bis] - Deering, D. and R. Hinden, "Internet Protocol, Version 6 - (IPv6) Specification", draft-ietf-6man-rfc2460bis-05 (work - in progress), June 2016. + Hinden, R., "Internet Protocol, Version 6 (IPv6) + Specification", draft-ietf-6man-rfc2460bis-06 (work in + progress), September 2016. [I-D.ietf-6man-segment-routing-header] Previdi, S., Filsfils, C., Field, B., Leung, I., Linkova, J., Aries, E., Kosugi, T., Vyncke, E., and D. Lebrun, "IPv6 Segment Routing Header (SRH)", draft-ietf-6man- segment-routing-header-01 (work in progress), March 2016. [I-D.ietf-bier-architecture] Wijnands, I., Rosen, E., Dolganow, A., Przygienda, T., and S. Aldrin, "Multicast using Bit Index Explicit Replication", draft-ietf-bier-architecture-04 (work in progress), July 2016. + [I-D.ietf-bier-mpls-encapsulation] + Wijnands, I., Rosen, E., Dolganow, A., Tantsura, J., + Aldrin, S., and I. Meilik, "Encapsulation for Bit Index + Explicit Replication in MPLS Networks", draft-ietf-bier- + mpls-encapsulation-05 (work in progress), July 2016. + [I-D.ietf-detnet-problem-statement] Finn, N. and P. Thubert, "Deterministic Networking Problem Statement", draft-ietf-detnet-problem-statement-00 (work in progress), April 2016. [I-D.ietf-mpls-residence-time] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S., and S. Vainshtein, "Residence Time Measurement in MPLS network", draft-ietf-mpls-residence-time-11 (work in progress), July 2016. [I-D.ietf-spring-segment-routing] Filsfils, C., Previdi, S., Decraene, B., Litkowski, S., and R. Shakir, "Segment Routing Architecture", draft-ietf- spring-segment-routing-09 (work in progress), July 2016. - [I-D.ietf-sunset4-gapanalysis] - Perreault, S., Tsou, T., Zhou, C., and P. Fan, "Gap - Analysis for IPv4 Sunset", draft-ietf- - sunset4-gapanalysis-07 (work in progress), April 2015. - [I-D.kumarzheng-bier-ping] Kumar, N., Pignataro, C., Akiya, N., Zheng, L., Chen, M., and G. Mirsky, "BIER Ping and Trace", draft-kumarzheng- bier-ping-03 (work in progress), July 2016. [I-D.mirsky-bier-path-mtu-discovery] Mirsky, G., Przygienda, T., and A. Dolganow, "Path Maximum Transmission Unit Discovery (PMTUD) for Bit Index Explicit Replication (BIER) Layer", draft-mirsky-bier-path-mtu- discovery-01 (work in progress), April 2016. [I-D.mirsky-bier-pmmm-oam] Mirsky, G., Zheng, L., Chen, M., and G. Fioccola, "Performance Measurement (PM) with Marking Method in Bit Index Explicit Replication (BIER) Layer", draft-mirsky- bier-pmmm-oam-01 (work in progress), March 2016. [I-D.ooamdt-rtgwg-oam-gap-analysis] Mirsky, G., Nordmark, E., Pignataro, C., Kumar, N., Kumar, - D., Chen, M., Yizhou, L., Mozes, D., Networks, J., and i. - ibagdona@gmail.com, "Operations, Administration and - Maintenance (OAM) for Overlay Networks: Gap Analysis", - draft-ooamdt-rtgwg-oam-gap-analysis-02 (work in progress), - July 2016. + D., Chen, M., Yizhou, L., Mozes, D., Networks, J., and I. + Bagdonas, "Operations, Administration and Maintenance + (OAM) for Overlay Networks: Gap Analysis", draft-ooamdt- + rtgwg-oam-gap-analysis-02 (work in progress), July 2016. [I-D.ooamdt-rtgwg-ooam-requirement] Kumar, N., Pignataro, C., Kumar, D., Mirsky, G., Chen, M., Nordmark, E., Networks, J., and D. Mozes, "Overlay OAM Requirements", draft-ooamdt-rtgwg-ooam-requirement-01 (work in progress), July 2016. - [IEEE802.1Qbv] - IEEE, "Enhancements for Scheduled Traffic", 2016, - . - [IEEE802.1Qca] IEEE 802.1, "IEEE 802.1Qca Bridges and Bridged Networks - Amendment 24: Path Control and Reservation", IEEE P802.1Qca/D2.1 P802.1Qca, June 2015, . - [IEEE802.1Qch] - IEEE, "Cyclic Queuing and Forwarding", 2016, - . - [IEEE8021CB] Finn, N., "Draft Standard for Local and metropolitan area networks - Seamless Redundancy", IEEE P802.1CB /D2.1 P802.1CB, December 2015, . [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, DOI 10.17487/RFC0791, September 1981, . @@ -2618,24 +2293,20 @@ [RFC7872] Gont, F., Linkova, J., Chown, T., and W. Liu, "Observations on the Dropping of Packets with IPv6 Extension Headers in the Real World", RFC 7872, DOI 10.17487/RFC7872, June 2016, . [ST20227] SMPTE 2022, "Seamless Protection Switching of SMPTE ST 2022 IP Datagrams", ST 2022-7:2013, 2013, . - [TSNTG] IEEE Standards Association, "IEEE 802.1 Time-Sensitive - Networks Task Group", 2013, - . - 10.2. URIs [1] http://6lab.cisco.com/stats/ [2] https://www.google.com/intl/en/ipv6/statistics.html [3] https://datatracker.ietf.org/wg/spring/charter/ [4] http://www.iana.org/assignments/g-ach-parameters/g-ach- parameters.xhtml