--- 1/draft-ietf-detnet-dp-sol-01.txt 2018-03-05 12:13:18.856166385 -0800 +++ 2/draft-ietf-detnet-dp-sol-02.txt 2018-03-05 12:13:18.940168373 -0800 @@ -1,30 +1,30 @@ DetNet J. Korhonen, Ed. Internet-Draft Nordic Intended status: Standards Track L. Andersson -Expires: August 2, 2018 Y. Jiang +Expires: September 6, 2018 Y. Jiang N. Finn Huawei B. Varga J. Farkas Ericsson CJ. Bernardos UC3M T. Mizrahi Marvell L. Berger LabN - January 29, 2018 + March 5, 2018 DetNet Data Plane Encapsulation - draft-ietf-detnet-dp-sol-01 + draft-ietf-detnet-dp-sol-02 Abstract This document specifies Deterministic Networking data plane encapsulation solutions. The described data plane solutions can be applied over either IP or MPLS Packet Switched Networks. Status of This Memo This Internet-Draft is submitted in full conformance with the @@ -33,21 +33,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on August 2, 2018. + This Internet-Draft will expire on September 6, 2018. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -61,71 +61,79 @@ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1. Terms used in this document . . . . . . . . . . . . . . . 4 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 4.1. DetNet data plane encapsulation requirements . . . . . . 8 4.2. Packet replication and elimination considerations . . . . 10 4.3. Packet reordering considerations . . . . . . . . . . . . 10 - 5. MPLS-based DetNet data plane solution . . . . . . . . . . . . 10 - 5.1. DetNet specific packet fields . . . . . . . . . . . . . . 10 - 5.2. Data plane encapsulation . . . . . . . . . . . . . . . . 11 - 5.3. DetNet control word . . . . . . . . . . . . . . . . . . . 12 - 5.4. Flow identification . . . . . . . . . . . . . . . . . . . 13 - 5.5. Service layer considerations . . . . . . . . . . . . . . 13 - 5.5.1. Edge node processing . . . . . . . . . . . . . . . . 13 - 5.5.2. Relay node processing . . . . . . . . . . . . . . . . 15 - 5.5.3. End system processing . . . . . . . . . . . . . . . . 17 - 5.6. Transport node considerations . . . . . . . . . . . . . . 17 - 5.6.1. Congestion protection . . . . . . . . . . . . . . . . 17 - 5.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 17 - 6. IPv6-based DetNet data plane solution . . . . . . . . . . . . 17 - 6.1. Data plane encapsulation . . . . . . . . . . . . . . . . 17 - 6.2. DetNet destination option . . . . . . . . . . . . . . . . 19 - 6.3. Flow identification . . . . . . . . . . . . . . . . . . . 20 - 6.4. Service layer considerations . . . . . . . . . . . . . . 20 - 6.4.1. Edge node processing . . . . . . . . . . . . . . . . 21 - 6.4.2. Relay node processing . . . . . . . . . . . . . . . . 23 - 6.4.3. End system processing . . . . . . . . . . . . . . . . 23 - 6.5. Transport node processing . . . . . . . . . . . . . . . . 23 - 6.5.1. Congestion protection . . . . . . . . . . . . . . . . 23 - 6.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 24 - 7. Other DetNet data plane considerations . . . . . . . . . . . 24 - 7.1. Class of Service . . . . . . . . . . . . . . . . . . . . 24 - 7.2. Quality of Service . . . . . . . . . . . . . . . . . . . 24 - 7.3. Cross-DetNet flow resource aggregation . . . . . . . . . 26 - 7.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 27 - 7.5. Layer 2 addressing and QoS Considerations . . . . . . . . 27 - 7.6. Interworking between MPLS- and IPv6-based encapsulations 28 - 7.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 28 - 8. Time synchronization . . . . . . . . . . . . . . . . . . . . 28 - 9. Management and control considerations . . . . . . . . . . . . 30 - 9.1. MPLS-based data plane . . . . . . . . . . . . . . . . . . 30 - 9.1.1. S-Label assignment and distribution . . . . . . . . . 30 - 9.1.2. Explicit routes . . . . . . . . . . . . . . . . . . . 30 - 9.2. IPv6-based data plane . . . . . . . . . . . . . . . . . . 30 - 9.2.1. Flow Label assignment and distribution . . . . . . . 30 - 9.2.2. Explicit routes . . . . . . . . . . . . . . . . . . . 31 - 9.3. Packet replication and elimination . . . . . . . . . . . 31 - 9.4. Congestion protection and latency control . . . . . . . . 31 - 9.5. Flow aggregation control . . . . . . . . . . . . . . . . 31 - 10. Security considerations . . . . . . . . . . . . . . . . . . . 31 - 11. IANA considerations . . . . . . . . . . . . . . . . . . . . . 31 - 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31 - 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 32 - 13.1. Normative references . . . . . . . . . . . . . . . . . . 32 - 13.2. Informative references . . . . . . . . . . . . . . . . . 34 - Appendix A. Example of DetNet data plane operation . . . . . . . 35 - Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 36 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 36 + 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 10 + 5.1. End-system specific considerations . . . . . . . . . . . 10 + 5.2. DetNet domain specific considerations . . . . . . . . . . 12 + 5.2.1. DetNet Bridging Service . . . . . . . . . . . . . . . 13 + 5.2.2. DetNet Routing Service . . . . . . . . . . . . . . . 14 + 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 16 + 5.3.1. Networks with multiple technology segments . . . . . 16 + 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 17 + 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 18 + 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 18 + 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 18 + 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 19 + 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 20 + 6.5. Service layer considerations . . . . . . . . . . . . . . 20 + 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 21 + 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 22 + 6.5.3. End system processing . . . . . . . . . . . . . . . . 24 + 6.6. Transport node considerations . . . . . . . . . . . . . . 24 + 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 24 + 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 24 + 7. IPv6-based DetNet data plane solution . . . . . . . . . . . . 24 + 7.1. Data plane encapsulation . . . . . . . . . . . . . . . . 24 + 7.2. DetNet destination option . . . . . . . . . . . . . . . . 26 + 7.3. Flow identification . . . . . . . . . . . . . . . . . . . 27 + 7.4. Service layer considerations . . . . . . . . . . . . . . 27 + 7.4.1. Edge node processing . . . . . . . . . . . . . . . . 28 + 7.4.2. Relay node processing . . . . . . . . . . . . . . . . 30 + 7.4.3. End system processing . . . . . . . . . . . . . . . . 30 + 7.5. Transport node processing . . . . . . . . . . . . . . . . 30 + 7.5.1. Congestion protection . . . . . . . . . . . . . . . . 30 + 7.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 31 + 8. Other DetNet data plane considerations . . . . . . . . . . . 31 + 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 31 + 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 31 + 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 33 + 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 34 + 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 34 + 8.6. Interworking between MPLS- and IPv6-based encapsulations 35 + 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 35 + 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 35 + 10. Management and control considerations . . . . . . . . . . . . 37 + 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 37 + 10.1.1. S-Label assignment and distribution . . . . . . . . 37 + 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 37 + 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 37 + 10.2.1. Flow Label assignment and distribution . . . . . . . 37 + 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 38 + 10.3. Packet replication and elimination . . . . . . . . . . . 38 + 10.4. Congestion protection and latency control . . . . . . . 38 + 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 38 + 11. Security considerations . . . . . . . . . . . . . . . . . . . 38 + 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 38 + 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 38 + 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 + 14.1. Normative references . . . . . . . . . . . . . . . . . . 39 + 14.2. Informative references . . . . . . . . . . . . . . . . . 41 + Appendix A. Example of DetNet data plane operation . . . . . . . 42 + Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 43 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 1. Introduction Deterministic Networking (DetNet) is a service that can be offered by a network to DetNet flows. DetNet provides these flows extremely low packet loss rates and assured maximum end-to-end delivery latency. General background and concepts of DetNet can be found in [I-D.ietf-detnet-architecture]. This document specifies the DetNet data plane and the on-wire @@ -438,43 +446,331 @@ 4.3. Packet reordering considerations DetNet service layer introduces also packet reordering functionality for use in DetNet edge and relay node and end system packet processing. The reordering functionality MAY be enabled in a DetNet node. The reordeing functionality relies on a presence of sequence numbers in a DetNet (compound) flows. The reordering processing is only applied to packets with a positive flow identification at the DetNet service layer. -5. MPLS-based DetNet data plane solution +5. DetNet encapsulation -5.1. DetNet specific packet fields +5.1. End-system specific considerations + + Data-flows requiring DetNet service are generated and terminated on + end-systems. Encapsulation depends on application and its + preferences. In a DetNet (or even a TSN) domain the DN (TSN) + functions use at most two flow parameters, namely Flow-ID and + Seq.Number. However, an application may exchange further flow + related parameters (e.g., time-stamp), which are not considered by DN + functions. + + Two types of end-systems are distinguished: + + o L3 (IP) end-system: application over L3 + + o L2 (Ethernet) end-system: application directly over L2 + + In case of Ethernet end-systems the application data is encapsulated + directly in L2. From the DN domain perspective no upper layer + protocols are visible. The Data-flow uses only Ethernet tag(s) and + further flow specific parameters (if needed) are hidden inside the + PDU. + + The IP end-system scenario is different. Data-flows are encapsulated + directly in L3 (i.e., IP) and the application may use further upper + layer protocols (e.g., RTP). Many valid combinations exist, and it + may be application specific how the IP header fields are used. Also, + usage of further upper layer protocols depends on application + requirements (e.g., time-stamp). Some examples for encoding of Flow- + ID or Seq.Number attributes: IP address, IPv6-Flow-label, L4 ports, + RTP-header, etc. + + As a general rule, DetNet domains MUST be capable to forward any + Data-flows and the DetNet domain MUST NOT intend to mandate end- + system encapsulation format. + + Furthermore, no application-level-proxy function is envisioned inside + the DetNet domain, so end-systems peer with end-systems using the + same application encapsulation format (see figure below): + + o L3 end-systems peer with L3 end-systems and + + o L2 end-systems peer with L2 end-systems + +-----+ + | X | +-----+ + +-----+ | X | + | Eth | ________ +-----+ + +-----+ _____ / \ | Eth | + \ / \__/ \___ +-----+ + \ / \ / + 0======== tunnel-1 ========0_ + | \ + \ | + 0========= tunnel-2 =========0 + / \ __/ \ + +-----+ \__ DetNet domain / \ + | X | \ __ / +-----+ + +-----+ \_______/ \_____/ | X | + | IP | +-----+ + +-----+ | IP | + +-----+ + + Figure 4: End-systems and the DetNet domain + +5.2. DetNet domain specific considerations + + From connection type perspective three scenarios are distinguished: + + 1. Directly attached: end-system is directly connected to an edge + node + + 2. Indirectly attached: end-system is behind a (L2-TSN / L3-DetNet) + sub-net + + 3. DN integrated: end-system is part of the DetNet domain + + L3 end-systems may use any of these connection types, however L2 end- + systems may use only the first two (directly or indirectly attached). + DetNet domain MUST allow communication between any end-systems of the + same type (L2-L2, L3-L3), independent of their connection type and + DetNet capability. However directly attached and indirectly attached + end-systems have no knowledge about the DetNet domain and its + encapsulation format at all. See the figure below for L3 end-system + scenarios. + + ____+----+ + +----+ _____ / | ES3| + | ES1|____ / \__/ +----+___ + +----+ \ / \ + + | + ____ \ _/ + +----+ __/ \ +__ DetNet domain / + | ES2|____/ L2/L3 |___/ \ __ __/ + +----+ \_______/ \_______/ \___/ + + Figure 5: Connection types of L3 end-systems + +5.2.1. DetNet Bridging Service + + The simplest DetNet service is to provide bridging (i.e., tunneling + for L2), where the connected hosts are in the same broadcast (BC) + domain. Forwarding over the DetNet domain is based on L2 (MAC) + addresses (i.e. dst-MAC), so L2 headers MUST be kept. For both IP + and MPLS PSN a DetNet specific tunnel encapsulation MUST be + introduced. + + +------+ + | X | + +------+ +------+ + | X | | IP | + +------+ +------+ + End+system | L2 | | L2 | + +-----+======+-+======+--+======+-+======++ + DetNet tunnel | IP | | MPLS | + +------+ +------+ + | L2 | | L2 | + +------+ +------+ + + Examples + + +------+ +------+ + | X | | X | + +------+ +------+ +------+ +------+ + | X | | X | | IP | | IP | + +------+ +------+ +------+ +------+ + | L2 | | L2 | | L2 | | L2 | + ++======+-+======+--+======+-+======++ + | IP | | MPLS | | IP | | MPLS | + +------+ +------+ +------+ +------+ + | L2 | | L2 | | L2 | | L2 | + +------+ +------+ +------+ +------+ + + Figure 6: Encapsulation format for DetNet Bridging + + As shown on the figure both L2 and L3 end-systems can be served by + such a DetNet Bridging service. + +5.2.2. DetNet Routing Service + + DetNet Routing service provides routing, therefore available only for + L3 hosts that are in different BC domains. Forwarding over the + DetNet domain is based on L3 (IP) addresses (i.e. dst-IP). + +5.2.2.1. MPLS PSN + + In case of an MPLS PSN at the ingress/egress (i.e., PE nodes of + DetNet domain) the IP packets are encapsulated in MPLS. The data- + flow IP header MUST be preserved as-is. + + +------+ +------+ + | X | | X | + +------+ +------+ + End+system | IP | | IP | + -----+------+-------+======+--- --+======+-- + DetNet | MPLS | | MPLS | + +------+ +------+ + | L2 | | L2 | + +------+ +------+ + + Figure 7: Encapsulation format for DetNet Routing in MPLS PSN for L3 + end-systems + +5.2.2.2. IP PSN + + In case of an IP PSN the same tunneling concept can be used as for an + MPLS PSN, but the tunnel is constructed by a new IP header (and + possible upper layer fields). The data-flow IP header MUST be + preserved as-is. + + +------+ +------+ + | X | | X | + +======+ +------+ + End-system | IP | | IP | + -----+------+-------+======+--- --+======+-- + DetNet | IP | | IP | + +------+ +------+ + | L2 | | L2 | + +------+ +------+ + + Figure 8: Encapsulation format for DetNet Routing in IP PSN for L3 + end-systems + + DetNet IP header contains the IP addresses of the ingress/egress PE + nodes of DetNet domain. The End-system IP header contains the IP + addresses of the end-systems. + + Note: In case of IP PSN one may consider avoiding the additional IP + encapsulation, however there are many issues with such an approach. + First, the DetNet nodes MUST be able to extract from the IP header + (and maybe upper layers) the attributes required by DetNet functions + (i.e. Flow-ID, Seq.Number). The challenge is that encoding of those + attributes may be application specific, so DetNet nodes MUST be + prepared to handle all application specific formats. Second, adding + further fields (e.g., explicit path information) to an existing IP + header may be impossible (e.g., due to security/encryption). + + Furthermore, DetNet domain IP-header format may collide with IP- + header format used by the source of a flow. Implementing such an + approach requires that source encapsulation is in-line with DetNet + domain encapsulation format, however we do not intend to mandate end- + systems' encapsulation format (see former text: As a general rule, + DetNet domains MUST be capable to forward any Data-flows and the + DetNet domain MUST NOT intend to mandate end-system encapsulation + format.). + +5.3. DetNet Inter-Working Function (DN-IWF) + +5.3.1. Networks with multiple technology segments + + There are network scenarios, where the DetNet domain contains + multiple technology segments (IP, MPLS) and all those segments are + under the same administrative control. Furthermore, DetNet nodes may + be interconnected via TSN segments. + + An important aspect of DetNet network design is placement of DetNet + functions across the domain. Designs based on segment-by-segment + optimization can provide only suboptimal solutions. In order to + achieve global optimum Inter-Working Functions (DN-IWF) can be placed + at segment border nodes, which stich together DetNet flows across + connected segments. + + DN-IWF may ensure that flow attributes are correlated across segment + borders. For example, there are two DetNet functions which require + Seq.Numbers: (1) Elimination: removes duplications from flows and (2) + IOD: ensures in-order-delivery of packet in a flow. Stitching flows + together and correlating attributes means for example that + replication of packets can happen in one segment and elimination of + duplicates in a different one. + + ______ + _____ / \__ + ____ / \__/ \___ ______ ++----+ __/ +======+ +==+ \ +----+ +|src |__/ Seg1 ) | | \ Seg3 \____| dst| ++----+ \_______+ \ Segment-2 | \+_____/ +----+ + \======+__ _+===/ + \ __ __/ + \_______/ \___/ + + +------------+ + +------------------E----+ | | ++----+ | | +----R---+ | +----+ +|src |-------R +---+ | E----------+ dst| ++----+ | | E--------+ +----+ + +--------------R | + +-----------------+ + + Figure 9: Optimal replication and elimination placement across + technology segments example + +5.3.2. DN-IWF related considerations + + The ultimate goal of DN-IWF is to (1) match and (2) translate segment + specific flow attributes. The DN-IWF ensures that segment specific + attributes comprise per domain unique attributes for the whole DetNet + domain. This characteristic can ensure that DetNet functions can be + based on per domain attributes and not per segment attributes. + + The two DetNet specific attributes have the following + characteristics: + + o Flow-ID: it is same in all packets of a flow + + o Seq.Number: it is different packet-by-packet + + For the Flow-ID the DN-IWF can implement a static mapping. The + situation is more complicated for Seq.Number as it is different + packet-by-packet, so it may need more sophisticated translation + unless its format is exactly the same in the two technology segments. + In this later case the DN-IWF can simple copy the Seq.Number field + between the tunneling encapsulation of the two technology segments. + + In case of three technology segments (IP, MPLS and TSN) three DN-IWF + functions can be specified. In the rest of this section the focus is + on the (1) IP - MPLS network scenario. Note: the use-cases are out- + of-scope for (2) TSN - IP, (3) TSN - MPLS. Note2: incompatible + format of Seq.Number with TSN. + + Simplest implementation of DN-IWF is provided if the flow attributes + have the same format. Such a common denominator of the tunnel + encapsulation format is the pseudowire encapsulation over both IP and + MPLS. + + Placeholder + + Figure 10: FIGURE Placeholder PW over X + +6. MPLS-based DetNet data plane solution + +6.1. DetNet specific packet fields The DetNet data plane encapsulation MUST include two DetNet specific information elements in each packet of a DetNet flow: (1) a flow identification and (2) a sequence number. The DetNet data plane encapsulation may consists further elements used for overlay tunneling, to distinguish between DetNet member flows of the same DetNet compound flow or to support OAM functions. -5.2. Data plane encapsulation +6.2. Data plane encapsulation - Figure 4 illustrates a DetNet data plane MPLS encapsulation. The + Figure 11 illustrates a DetNet data plane MPLS encapsulation. The MPLS-based encapsulation of the DetNet flows is a good fit for the Layer-2 interconnect deployment cases (see Figure 1). Furthermore, end to end DetNet service i.e., native DetNet deployment (see Figure 2) is also possible if DetNet end systems are capable of initiating and termination MPLS encapsulated packets. Transport of - IP encapsulated DetNet flows, see Section 6, over MPLS-based DetNet + IP encapsulated DetNet flows, see Section 7, over MPLS-based DetNet data plane is also possible. Interworking between PW- and IPv6-based - encapsulations is discussed further in Section 7.6. + encapsulations is discussed further in Section 8.6. The MPLS-based DetNet data plane encapsulation consists of: o DetNet control word (d-CW) containing sequencing information for packet replication and duplicate elimination purposes. There MUST a separate sequence number space for each DetNet flow. o DetNet Label that identifies a DetNet flow within a DetNet Edge or a Relay node. The DetNet label MUST be at the bottom of the label stack. @@ -504,60 +800,60 @@ +---------------------------------+ +--> DetNet data plane | S-Label | | MPLS encapsulation +---------------------------------+ <--/ | T-Label(s) | +---------------------------------+ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+ - Figure 4: Encapsulation of a DetNet flow in an MPLS(-TP) PSN + Figure 11: Encapsulation of a DetNet flow in an MPLS(-TP) PSN -5.3. DetNet control word +6.3. DetNet control word A DetNet control word (d-CW) conforms to the Generic PW MPLS Control - Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 5. + Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 12. The upper nibble of the d-CW MUST be set to zero (0). The effective sequence number bit length is between 0 and 28 bits, and configured either by a control plane or manually for each DetNet flow. The sequence number is aligned to the right (least significant bits) and unused bits MUST be set to zero (0). Each DetNet flow MUST have its own sequence number counter. The sequence number is incremented by one for each new packet. The d-CW MUST always be present in a packet. In a case the sequence number is not used (e.g., for DetNet-t-flows) the control plane or the manual configuration has to define zero (0) bit length seqeunce number and the value of the sequence number MUST be set to zero (0). 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0| Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Figure 5: DetNet Control Word + Figure 12: DetNet Control Word -5.4. Flow identification +6.4. Flow identification DetNet flow identification at a DetNet service layer is realized by an S-label. It maps a Detnet flow to a specific d-CW in a DetNet node. The S-label used for flow identification MUST be bottom label of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede the d-CW. An S-label for a single DetNet flow does not need to be unique DetNet domain wide. As long as two or more different DetNet flows do not errorneously map to a same d-CW in a DetNet node the labels may vary. -5.5. Service layer considerations +6.5. Service layer considerations [Editor's note: quite a bit of unfinished and old text in the following sections.] The edge and relay node internal procedures of the PREF are implementation specific. The order of a packet elimination or replication is out of scope in this specification. However, care should be taken that the replication function does not actually loopback packets as "replicas". Looped back packets include artificial delay when the node that originally initiated the packet @@ -568,29 +864,30 @@ Comment #29: SB> There needs to be some text about preventing a node ever receiving its own replicated packets. Indeed that would suggest that the flow id should be changed and replication should only take place on configured flow IDs. I have a feeling that this would all be a lot safer if replication only happened at ingress and we managed the diversity of the paths. Discussion: Agree on hardening the loopback text considerations. -5.5.1. Edge node processing +6.5.1. Edge node processing TBD. [Editor's note: Since we are not defining the inner workings and implementation of the DetNet Egde node - rather only what goes in and what comes out, and of course the on-wire details, then the figures shown in the coming section would not need to detail the inner architecture of a DetNet Node.] + +---------------------------------------+ | DetNet Edge Node | +---------------------------------------+ | | | | | | | | Client AC | +---o <-------> o o<----------> | Elim. | | | | <---------->o <-------| | +-------------+ | | | | | | +---o <-------> o | @@ -601,65 +898,64 @@ Client AC | | Repl. | | | <---------->o o <-----X-> o o<----------> | | Elim. | | +-------------+ +-------------+ | | | | Client AC | | | | <---------->o o <-------> o o<----------> | | | | +---------------------------------------+ - Figure 6: DetNet Edge Node processing + Figure 13: DetNet Edge Node processing An edge node participates to the packet replication and duplication elimination. Required processing is done within an extended forwarder function. In the case the native service processing (NSP) is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and duplicate elimination MAY entirely be done in the NSP and bypassing the DetNet flow encapsulation and logic entirely, and thus is able to operate over unmodified implementation and deployment. The NSP approach works only between edge nodes and cannot make use of relay - nodes (see Section 5.5.2). + nodes (see Section 6.5.2). Comment #31 SB> This would be a fine way to operate the PW system - edge to edge. Discussion: When it comes to use of NSPs, agree. Also for "island interconnect" this is a fine. However, when there is a need to do PREF in a middle, plain edge to edge is not enough. The DetNet-aware extended forwarder selects the egress DetNet member flow based on the DetNet forwarding rules. In both "normal AC" and "Packet AC" cases there may be no DetNet encapsulation header - available yet as it is the case with relay nodes (see Section 5.5.2). - + available yet as it is the case with relay nodes (see Section 6.5.2). It is the responsibility of the extended forwarder within the edge node to push the DetNet specific encapsulation (if not already present) to the packet before forwarding it to the appropriate egress DetNet member flow instance(s). Comment #32 SB> I am not convinced of the wisdom of having a mid- point node convert a flow into a DN flow, which is what you are implying here. This seems like an ingress function. Discussion: OK. The text here has issues and seems to mix relay and edge. The extended forwarder MAY copy the sequencing information from the native DetNet packet into the DetNet sequence number field and vice versa. If there is no existing sequencing information available in the native packet or the forwarder chose not to copy it from the native packet, then the extended forwarder MUST maintain a sequence number counter for each DetNet flow (indexed by the DetNet flow identification). -5.5.2. Relay node processing +6.5.2. Relay node processing TBD. A DetNet Relay node participates to the packet replication and duplication elimination. This processing is done within an extended forwarder function. Whether an ingress DetNet member flow receives DetNet specific processing depends on how the forwarding is programmed. For some DetNet member flows the relay node can act as a normal relay node and for some apply the DetNet specific processing (i.e., PREF). @@ -668,21 +964,21 @@ sure what it does in the absence of a PREF function. Discussion: Relay node was a DetNet aware S-PE originally, which is not explicitly stated here anymore, thus slightly confusing text here. The text here needs to clarify the roles of PREF and switching functions. A DetNet relay is described in the architecture document. However, there is definitely room for termonilogy and text improvements. It is also possible to treat the relay node as a transit node, see - Section 7.3. Again, this is entirely up to how the forwarding has + Section 8.3. Again, this is entirely up to how the forwarding has been programmed. The DetNet-aware forwarder selects the egress DetNet member flow segment based on the flow identification. The mapping of ingress DetNet member flow segment to egress DetNet member flow segment may be statically or dynamically configured. Additionally the DetNet- aware forwarder does duplicate frame elimination based on the flow identification and the sequence number combination. The packet replication is also done within the DetNet-aware forwarder. During elimination and the replication process the sequence number of the @@ -706,50 +1002,50 @@ | | Repl. | | | ----------->o o ------+-> o o-----------> | | | | Ingress +-------------+ +-------------+ | | | | Egress | | | | ----------->o o --------> o o-----------> | | | | +---------------------------------------+ - Figure 7: DetNet Relay Node processing + Figure 14: DetNet Relay Node processing Comment #35 SB> Somewhere in the dp document there needs to be a note of the requirement for interfaces to do fast exchange of counter state, and a note to those planning the network and designing the control plane that they need to provide support for this. Discussion: We kinf of agree but also think the above exchange or synchronization of counter states is not in our scope to solve. -5.5.3. End system processing +6.5.3. End system processing TBD. -5.6. Transport node considerations +6.6. Transport node considerations -5.6.1. Congestion protection +6.6.1. Congestion protection TBD. -5.6.2. Explicit routes +6.6.2. Explicit routes TBD. -6. IPv6-based DetNet data plane solution +7. IPv6-based DetNet data plane solution -6.1. Data plane encapsulation +7.1. Data plane encapsulation - Figure 8 illustrates a DetNet native IPv6 encapsulation. The native + Figure 15 illustrates a DetNet native IPv6 encapsulation. The native IPv6 encapsulation is meant for end to end Detnet service use cases, where the end stations are DetNet-aware (see Figure 3). Technically it is possible to use the IPv6 encapsulation to tunnel any traffic over a DetNet enabled network, which would make native IPv6 encapsulation also a valid data plane choice for an interconnect use case (see Figure 1). The native IPv6-based DetNet data plane encapsulation consists of: o IPv6 header as the transport protocol. @@ -796,24 +1092,24 @@ | DetNet Flow | | Payload | | | /---------------------------------\ H Optional DetNet DstOpt Hdr H \---------------------------------/ | IPv6 header | | (with set Flow label) | +---------------------------------+ - Figure 8: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow + Figure 15: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow without a routing header - Figure 9 illustrates an IPv6 packet for the case where a routing + Figure 16 illustrates an IPv6 packet for the case where a routing header has been added into the packet by a DetNet-aware end system (again assuming DetNet-s- or DetNet-st-flows). Note that the use of routing header such as the one with the segment routing option is not mandatory for explicit routes. Similar functionality can be arranged using other means as well (e.g., using policy routing or layer-2 means). +---------------------------------+ | | | DetNet Flow | @@ -823,98 +1119,98 @@ H DetNet DstOpt Hdr H \---------------------------------/ | Routing header | /---------------------------------\ H DetNet DstOpt Hdr H \---------------------------------/ | IPv6 header | | (with set Flow label) | +---------------------------------+ - Figure 9: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flows - with routing header + Figure 16: Encapsulation of a native IPv6 DetNet-s- or DetNet-st- + flows with routing header IPv6 extension headers can only be inserted by a node that initiated the IPv6 packet. IPv6 extension headers, except for the Hop-by-Hop Option headers, can only be processed by an IPv6 node that is identified by the Destination Address field of the IPv6 header (see Section 0 of [RFC8200]. Therefore, if a DetNet-aware end system only inserted the DetNet Destination Option into the IPv6 but e.g., a DetNet Edge node is configured to enforce an explicit route for the IPv6 packet using a source routing header, then it has no other possibility than add an outer tunneling IPv6 header with required extension headers in it. The processing of IPv6 packets in a DetNet - Edge node is discussed further in Section 6.4.1. + Edge node is discussed further in Section 7.4.1. -6.2. DetNet destination option +7.2. DetNet destination option A DetNet flow must carry sequencing information for packet replication and elimination function (PREF) purposes. This document specifies a new IPv6 Destination Option: the DetNet Destination Option for that purpose. The format of the option is illustrated in - Figure 10. + Figure 17. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TBD1 | 4 | 0 | 28 bit sequence +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Figure 10: DetNet Destination Option + Figure 17: DetNet Destination Option The Option Type for the DetNet Destination Option is set to TBD1. [To be removed from the final version of the document: The Option Type MUST have the two most significant bits set to 10b] If an IPv6 packet gets dropped due the DetNet Service layer processing based on the DetNet Destination Option an ICMPv6 packet of any type MUST NOT be sent back to the source of the packet. -6.3. Flow identification +7.3. Flow identification The DetNet flow identification is based on the IPv6 Flow Label and the source address combination. The two fields uniquelly identify the end to end native IPv6 encapsulated DetNet flow. Obviously, the identification fails if any intermediate node modifies either the source address or the Flow Label. Comment #27 SB> See earlier. If there are enough IPv6 addresses to address video fragments, why not DN flows? Then this problem goes away. Discussion: See the earlier comment #25 discussion. If nodes get their addressies via DHCPv6 basically ruins this mechanism. Also the assumption for this to work is that the node has a full /64 to use, which is not always the case. Otherwise the idea is just fine. -6.4. Service layer considerations +7.4. Service layer considerations [Editor's note: this section is TBD. It will detail the PREF functionality.] o PREF - requires both flow identification and sequence numbering. o Packet reordeing - requires both flow identification and sequence numbering. A DetNet service layer processing can be done at each DetNet node that matches the IPv6 header's Destination Address. Then, if the DetNet flow identification provides a positive match for the DetNet flow that the node has a service layer state installed e.g., for PREF or packet reordering purposes, further service layer processing takes place. In a case of PREF or packet reordering that means processing the DetNet Destination Option for the identified DetNet flow. -6.4.1. Edge node processing +7.4.1. Edge node processing [Editor's note: This is the start of the IPv6 handling text - there are errors and bad language. The founding assumption is the use of source routing when intermediate nodes (relays/edges) need to modify packets. This is due the text in RFC8200 and the fact that without hph options only routing+dsthdr is usable with intermediates under strict RFC8200.. ] [Editor's note: Regrading the source routing and the "example" SRv6 approach. Current text is based on the assumption that intermediates @@ -948,21 +1244,21 @@ at DetNet Edge nodes. Possible explicit routes between edge nodes are arranged by other than IPv6 specific means. 4. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress DetNet Edge node and multiple DetNet Relay nodes may process DetNet flow packets before reaching an egress DetNet Edge node. Explicit routes between edge nodes has to be arranged by IPv6 specific means. A generic DetNet IPv6 encapsulation for a DetNet flow packet between - DetNet Edge nodes is shown in Figure 11. Essentially every time an + DetNet Edge nodes is shown in Figure 18. Essentially every time an igress DetNet Edge node has to insert something into the DetNet flow packet it has to add an outer tunneling IPv6 header, which then contain possible additional extension headers. +---------------------------------+ | | | DetNet Flow | | Payload | | | +---------------------------------+ @@ -972,24 +1268,24 @@ | (with set Flow label) (1) | +---------------------------------+ <--+ | Optional Routing header | | +---------------------------------+ | | Optional DetNet DstOpt Hdr (2) | +-- Added/removed by +---------------------------------+ | DetNet Edge node | Outer IPv6 header | | | (with set Flow label) (2) | | +---------------------------------+ <--+ - Figure 11: Encapsulation of a DetNet-flow IPv6 packet at the DetNet + Figure 18: Encapsulation of a DetNet-flow IPv6 packet at the DetNet Edge node -6.4.1.1. Ingress DetNet Edge node processing +7.4.1.1. Ingress DetNet Edge node processing Case 1) MAY require an addition of the DetNet Destination Option if packet reordering is requested at the egress DetNet Edge node. Otherwise, no modifications except rewriting the IPv6 header flow label to the packet is done. If modifications are required then: o The outer IPv6 header is added with the Source Address set to the ingress DetNet Edge node address and the Destination Address set to the egress DetNet Edge node address. @@ -1020,38 +1316,38 @@ o A source routing header with addresses of those DetNet Relay nodes that must be traversed is inserted into the outer IPv6 header. Case 3) ...[Editor's note: is it OK if the sequece number added here by the edge node has only local significance between the edge nodes and not end to end between end systems? ] Case 4) ... -6.4.1.2. Engress DetNet Edge node processing +7.4.1.2. Engress DetNet Edge node processing -6.4.2. Relay node processing +7.4.2. Relay node processing TBD. -6.4.3. End system processing +7.4.3. End system processing TBD. -6.5. Transport node processing +7.5. Transport node processing -6.5.1. Congestion protection -6.5.2. Explicit routes +7.5.1. Congestion protection +7.5.2. Explicit routes -7. Other DetNet data plane considerations +8. Other DetNet data plane considerations -7.1. Class of Service +8.1. Class of Service Class and quality of service, i.e., CoS and QoS, are terms that are often used interchangeably and confused. In the context of DetNet, CoS is used to refer to mechanisms that provide traffic forwarding treatment based on aggregate group basis and QoS is used to refer to mechanisms that provide traffic forwarding treatment based on a specific DetNet flow basis. Examples of existing network level CoS mechanisms include DiffServ which is enabled by IP header differentiated services code point (DSCP) field [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p @@ -1072,35 +1368,35 @@ mechanisms. The 2-bit explicit congestion notification (ECN) [RFC3168] field MAY also be used. One additional consideration for DetNet nodes which support CoS services is that they MUST ensure that the CoS service classes do not impact the congestion protection and latency control mechanisms used to provide DetNet QoS. This requirement is similar to requirement for MPLS LSRs to that CoS LSPs do not impact the resources allocated to TE LSPs via [RFC3473]. -7.2. Quality of Service +8.2. Quality of Service Quality of Service (QoS) mechanisms for flow specific traffic treatment typically includes a guarantee/agreement for the service, and allocation of resources to support the service. Example QoS mechanisms include discrete resource allocation, admission control, flow identification and isolation, and sometimes path control, traffic protection, shaping, policing and remarking. Example protocols that support QoS control include Resource ReSerVation Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. The existing MPLS mechanisms defined to support CoS [RFC3270] can also be used to reserve resources for specific traffic classes. In addition to explicit routes, and packet replication and - elimination, described in Section 5 above, DetNet provides zero + elimination, described in Section 6 above, DetNet provides zero congestion loss and bounded latency and jitter. As described in [I-D.ietf-detnet-architecture], there are different mechanisms that maybe used separately or in combination to deliver a zero congestion loss service. These mechanisms are provided by the either the MPLS or IP layers, and may be combined with the mechanisms defined by the underlying network layer such as 802.1TSN. A baseline set of QoS capabilities for DetNet flows carried in PWs and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes @@ -1112,39 +1408,39 @@ support the full range of DetNet services. In all cases, the existing label-based marking mechanisms defined for TE-LSPs and even E-LSPs are use to support the identification of flows requiring DetNet QoS. QoS for DetNet flows carried in IPv6 MUST be provided locally by the DetNet-aware hosts and routers supporting DetNet flows. Such support will leverage the underlying network layer such as 802.1TSN. The traffic control mechanisms used to deliver QoS for IP encapsulated DetNet flows are expected to be defined in a future document. From - an encapsulation perspective, and as defined in Section 6, the + an encapsulation perspective, and as defined in Section 7, the combination of the Flow Label together with the IP source address uniquely identifies a DetNet flow. Packets that are marked with a DetNet Class of Service value, but that have not been the subject of a completed reservation, can disrupt the QoS offered to properly reserved DetNet flows by using resources allocated to the reserved flows. Therefore, the network nodes of a DetNet network MUST: o Defend the DetNet QoS by discarding or remarking (to a non-DetNet CoS) packets received that are not the subject of a completed reservation. o Not use a DetNet reserved resource, e.g. a queue or shaper reserved for DetNet flows, for any packet that does not carry a DetNet Class of Service marker. -7.3. Cross-DetNet flow resource aggregation +8.3. Cross-DetNet flow resource aggregation The ability to aggregate individual flows, and their associated resource control, into a larger aggregate is an important technique for improving scaling of control in the data, management and control planes. This document identifies the traffic identification related aspects of aggregation of DetNet flows. The resource control and management aspects of aggregation (including the queuing/shaping/ policing implications) will be covered in other documents. The data plane implications of aggregation are independent for PW/MPLS and IP encapsulated DetNet flows. @@ -1178,21 +1474,21 @@ Discussion: -- In both the MPLS and IP cases, additional details of the traffic control capabilities needed at a DetNet-aware node may be covered in the new service descriptions mentioned above or in separate future documents. Management and control plane mechanisms will also need to ensure that the service required on the aggregate flow (H-LSP or DSCP) are provided, which may include the discarding or remarking mentioned in the previous sections. -7.4. Bidirectional traffic +8.4. Bidirectional traffic Some DetNet applications generate bidirectional traffic. Using MPLS definitions [RFC5654] there are associated bidirectional flows, and co-routed bidirectional flows. MPLS defines a point-to-point associated bidirectional LSP as consisting of two unidirectional point-to-point LSPs, one from A to B and the other from B to A, which are regarded as providing a single logical bidirectional transport path. This would be analogous of standard IP routing, or PWs running over two reciprocal unidirection LSPs. MPLS defines a point-to-point co-routed bidirectional LSP as an associated bidirectional LSP which @@ -1209,21 +1505,21 @@ flows, there are no special bidirectional features with respect to the data plane other than need for the two directions take the same paths. Fate sharing and associated vs co-routed bidirectional flows can be managed at the control level. Note, that there is no stated requirement for bidirectional DetNet flows to be supported using the same IPv6 Flow Labels or MPLS Labels in each direction. Control mechanisms will need to support such bidirectional flows for both IPv6 and MPLS, but such mechanisms are out of scope of this document. An example control plane solution for MPLS can be found in [RFC7551]. -7.5. Layer 2 addressing and QoS Considerations +8.5. Layer 2 addressing and QoS Considerations The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 Working Group have defined (and are defining) a number of amendments to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and bounded latency in bridged networks. IEEE 802.1CB [IEEE8021CB] defines packet replication and elimination functions that should prove both compatible with and useful to, DetNet networks. As is the case for DetNet, a Layer 2 network node such as a bridge may need to identify the specific DetNet flow to which a packet @@ -1239,31 +1535,31 @@ within the bridged network over which it is carried. Furthermore, IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet sequence number can be encoded in an Ethernet frame. Ensuring that the proper Ethernet VLAN tag priority and destination MAC address are used on a DetNet/TSN packet may require further clarification of the customary L2/L3 transformations carried out by routers and edge label switches. Edge nodes may also have to move sequence number fields among Layer 2, PW, and IPv6 encapsulations. -7.6. Interworking between MPLS- and IPv6-based encapsulations +8.6. Interworking between MPLS- and IPv6-based encapsulations [Editor's note: add considerations for interworking between MPLS- based and native IPv6-based DetNet encapsuations.] -7.7. IPv4 considerations +8.7. IPv4 considerations [Editor's note: The fact is that there are and will be deployments using IPv4. Neglecting it entirely is not feasible.] -8. Time synchronization +9. Time synchronization Comment #39 SB> This section should point the reader to RFC8169 (residence time in MPLS n/w. We need to consider if we need to introduce the same concept in IP. Discussion: Agree. For IP we could reference to PTPv2 or v3 over UDP/IP, since it measures residence time among other things. [Editor's note: describe a bit of issues and deployment considerations related to time-synchronization within DetNet. Refer @@ -1318,104 +1614,104 @@ synchronization accuracy, since the observed path delay will be bivalent. Comment #40 SB> I am not sure why we sould not use PREP. We should explain to the reader. Discussion: Agree that a this can be opened a bit more in detail. The issue is explained briefly in the last sentence but it could be more clear. -9. Management and control considerations +10. Management and control considerations [Editor's note: This section needs to be different for MPLS and IPv6 solutions. Most solutions are technology dependant,] While management plane and control planes are traditionally considered separately, from the Data Plane perspective there is no practical difference based on the origin of flow provisioning information. This document therefore does not distinguish between information provided by a control plane protocol, e.g., RSVP-TE [RFC3209] and [RFC3473], or by a network management mechanisms, e.g., RestConf [RFC8040] and YANG [RFC7950]. [Editor's note: This section is a work in progress. discuss here what kind of enhancements are needed for DetNet and specifically for PREF and DetNet zero congest loss and latency control. Need to cover both traffic control (queuing) and connection control (control plane).] -9.1. MPLS-based data plane +10.1. MPLS-based data plane -9.1.1. S-Label assignment and distribution +10.1.1. S-Label assignment and distribution [Editor's note: Outdated and MPLS specific.. and needs more work.] The DetNet S-Label distribution follows the same mechanisms specified for XYZ . The details of the control plane protocol solution required for the label distribution and the management of the label number space are out of scope of this document. -9.1.2. Explicit routes +10.1.2. Explicit routes [Editor's note: Outdated.. and needs more work.] [TBD: based on MPLS TE and possibly IPv6 SR] -9.2. IPv6-based data plane +10.2. IPv6-based data plane -9.2.1. Flow Label assignment and distribution +10.2.1. Flow Label assignment and distribution [Editor's note: Outdated and IPv6 Specific.. and needs more work.] The IPv6 Flow Label distribution and the label number space are out of scope of this document. However, it should be noted that the combination of the IPv6 source address and the IPv6 Flow Label is assumed to be unique within the DetNet-enabled network. Therefore, as long as each node is able to assign unique Flow Labels for the source address(es) it is using the DetNet-enabled network wide flow identification uniqueness is guaranteed. -9.2.2. Explicit routes +10.2.2. Explicit routes [Editor's note: Outdated.. and needs more work.] [TBD: What we have there for IPv6 and explicit routes] -9.3. Packet replication and elimination +10.3. Packet replication and elimination [Editor's note: Outdated and at the functional level technology independent.. but needs more work.] The control plane protocol solution required for managing the PREF processing is outside the scope of this document. -9.4. Congestion protection and latency control +10.4. Congestion protection and latency control [TBD] -9.5. Flow aggregation control +10.5. Flow aggregation control [TBD] -10. Security considerations +11. Security considerations The security considerations of DetNet in general are discussed in [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other security considerations will be added in a future version of this draft. -11. IANA considerations +12. IANA considerations TBD. -12. Acknowledgements +13. Acknowledgements The author(s) ACK and NACK. The following people were part of the DetNet Data Plane Solution Design Team: Jouni Korhonen Janos Farkas @@ -1432,23 +1728,23 @@ Carlos J. Bernardos The DetNet chairs serving during the DetNet Data Plane Solution Design Team: Lou Berger Pat Thaler -13. References +14. References -13.1. Normative references +14.1. Normative references [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2211] Wroclawski, J., "Specification of the Controlled-Load Network Element Service", RFC 2211, DOI 10.17487/RFC2211, September 1997, . @@ -1522,21 +1818,21 @@ [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. Aissaoui, "Segmented Pseudowire", RFC 6073, DOI 10.17487/RFC6073, January 2011, . [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017, . -13.2. Informative references +14.2. Informative references [I-D.ietf-6man-segment-routing-header] Previdi, S., Filsfils, C., Raza, K., Dukes, D., Leddy, J., Field, B., daniel.voyer@bell.ca, d., daniel.bernier@bell.ca, d., Matsushima, S., Leung, I., Linkova, J., Aries, E., Kosugi, T., Vyncke, E., Lebrun, D., Steinberg, D., and R. Raszuk, "IPv6 Segment Routing Header (SRH)", draft-ietf-6man-segment-routing-header-08 (work in progress), January 2018.