--- 1/draft-ietf-detnet-dp-sol-02.txt 2018-03-05 16:13:14.992185646 -0800 +++ 2/draft-ietf-detnet-dp-sol-03.txt 2018-03-05 16:13:15.088187947 -0800 @@ -10,21 +10,21 @@ Ericsson CJ. Bernardos UC3M T. Mizrahi Marvell L. Berger LabN March 5, 2018 DetNet Data Plane Encapsulation - draft-ietf-detnet-dp-sol-02 + draft-ietf-detnet-dp-sol-03 Abstract This document specifies Deterministic Networking data plane encapsulation solutions. The described data plane solutions can be applied over either IP or MPLS Packet Switched Networks. Status of This Memo This Internet-Draft is submitted in full conformance with the @@ -66,74 +66,74 @@ 3. Requirements language . . . . . . . . . . . . . . . . . . . . 6 4. DetNet data plane overview . . . . . . . . . . . . . . . . . 6 4.1. DetNet data plane encapsulation requirements . . . . . . 8 4.2. Packet replication and elimination considerations . . . . 10 4.3. Packet reordering considerations . . . . . . . . . . . . 10 5. DetNet encapsulation . . . . . . . . . . . . . . . . . . . . 10 5.1. End-system specific considerations . . . . . . . . . . . 10 5.2. DetNet domain specific considerations . . . . . . . . . . 12 5.2.1. DetNet Bridging Service . . . . . . . . . . . . . . . 13 5.2.2. DetNet Routing Service . . . . . . . . . . . . . . . 14 - 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 16 - 5.3.1. Networks with multiple technology segments . . . . . 16 - 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 17 - 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 18 - 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 18 - 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 18 - 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 19 - 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 20 - 6.5. Service layer considerations . . . . . . . . . . . . . . 20 - 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 21 - 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 22 - 6.5.3. End system processing . . . . . . . . . . . . . . . . 24 - 6.6. Transport node considerations . . . . . . . . . . . . . . 24 - 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 24 - 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 24 - 7. IPv6-based DetNet data plane solution . . . . . . . . . . . . 24 - 7.1. Data plane encapsulation . . . . . . . . . . . . . . . . 24 - 7.2. DetNet destination option . . . . . . . . . . . . . . . . 26 - 7.3. Flow identification . . . . . . . . . . . . . . . . . . . 27 - 7.4. Service layer considerations . . . . . . . . . . . . . . 27 - 7.4.1. Edge node processing . . . . . . . . . . . . . . . . 28 - 7.4.2. Relay node processing . . . . . . . . . . . . . . . . 30 - 7.4.3. End system processing . . . . . . . . . . . . . . . . 30 - 7.5. Transport node processing . . . . . . . . . . . . . . . . 30 - 7.5.1. Congestion protection . . . . . . . . . . . . . . . . 30 - 7.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 31 - 8. Other DetNet data plane considerations . . . . . . . . . . . 31 - 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 31 - 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 31 - 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 33 - 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 34 - 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 34 - 8.6. Interworking between MPLS- and IPv6-based encapsulations 35 - 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 35 - 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 35 - 10. Management and control considerations . . . . . . . . . . . . 37 - 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 37 - 10.1.1. S-Label assignment and distribution . . . . . . . . 37 - 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 37 - 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 37 - 10.2.1. Flow Label assignment and distribution . . . . . . . 37 - 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 38 - 10.3. Packet replication and elimination . . . . . . . . . . . 38 - 10.4. Congestion protection and latency control . . . . . . . 38 - 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 38 - 11. Security considerations . . . . . . . . . . . . . . . . . . . 38 - 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 38 - 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 38 - 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 - 14.1. Normative references . . . . . . . . . . . . . . . . . . 39 - 14.2. Informative references . . . . . . . . . . . . . . . . . 41 - Appendix A. Example of DetNet data plane operation . . . . . . . 42 - Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 43 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 + 5.3. DetNet Inter-Working Function (DN-IWF) . . . . . . . . . 17 + 5.3.1. Networks with multiple technology segments . . . . . 17 + 5.3.2. DN-IWF related considerations . . . . . . . . . . . . 18 + 6. MPLS-based DetNet data plane solution . . . . . . . . . . . . 19 + 6.1. DetNet specific packet fields . . . . . . . . . . . . . . 19 + 6.2. Data plane encapsulation . . . . . . . . . . . . . . . . 19 + 6.3. DetNet control word . . . . . . . . . . . . . . . . . . . 20 + 6.4. Flow identification . . . . . . . . . . . . . . . . . . . 21 + 6.5. Service layer considerations . . . . . . . . . . . . . . 21 + 6.5.1. Edge node processing . . . . . . . . . . . . . . . . 22 + 6.5.2. Relay node processing . . . . . . . . . . . . . . . . 23 + 6.5.3. End system processing . . . . . . . . . . . . . . . . 25 + 6.6. Transport node considerations . . . . . . . . . . . . . . 25 + 6.6.1. Congestion protection . . . . . . . . . . . . . . . . 25 + 6.6.2. Explicit routes . . . . . . . . . . . . . . . . . . . 25 + 7. IPv6-based DetNet data plane solution . . . . . . . . . . . . 25 + 7.1. Data plane encapsulation . . . . . . . . . . . . . . . . 25 + 7.2. DetNet destination option . . . . . . . . . . . . . . . . 27 + 7.3. Flow identification . . . . . . . . . . . . . . . . . . . 28 + 7.4. Service layer considerations . . . . . . . . . . . . . . 28 + 7.4.1. Edge node processing . . . . . . . . . . . . . . . . 29 + 7.4.2. Relay node processing . . . . . . . . . . . . . . . . 31 + 7.4.3. End system processing . . . . . . . . . . . . . . . . 31 + 7.5. Transport node processing . . . . . . . . . . . . . . . . 31 + 7.5.1. Congestion protection . . . . . . . . . . . . . . . . 31 + 7.5.2. Explicit routes . . . . . . . . . . . . . . . . . . . 32 + 8. Other DetNet data plane considerations . . . . . . . . . . . 32 + 8.1. Class of Service . . . . . . . . . . . . . . . . . . . . 32 + 8.2. Quality of Service . . . . . . . . . . . . . . . . . . . 32 + 8.3. Cross-DetNet flow resource aggregation . . . . . . . . . 34 + 8.4. Bidirectional traffic . . . . . . . . . . . . . . . . . . 35 + 8.5. Layer 2 addressing and QoS Considerations . . . . . . . . 35 + 8.6. Interworking between MPLS- and IPv6-based encapsulations 36 + 8.7. IPv4 considerations . . . . . . . . . . . . . . . . . . . 36 + 9. Time synchronization . . . . . . . . . . . . . . . . . . . . 36 + 10. Management and control considerations . . . . . . . . . . . . 38 + 10.1. MPLS-based data plane . . . . . . . . . . . . . . . . . 38 + 10.1.1. S-Label assignment and distribution . . . . . . . . 38 + 10.1.2. Explicit routes . . . . . . . . . . . . . . . . . . 38 + 10.2. IPv6-based data plane . . . . . . . . . . . . . . . . . 38 + 10.2.1. Flow Label assignment and distribution . . . . . . . 38 + 10.2.2. Explicit routes . . . . . . . . . . . . . . . . . . 39 + 10.3. Packet replication and elimination . . . . . . . . . . . 39 + 10.4. Congestion protection and latency control . . . . . . . 39 + 10.5. Flow aggregation control . . . . . . . . . . . . . . . . 39 + 11. Security considerations . . . . . . . . . . . . . . . . . . . 39 + 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 39 + 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 39 + 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 40 + 14.1. Normative references . . . . . . . . . . . . . . . . . . 40 + 14.2. Informative references . . . . . . . . . . . . . . . . . 42 + Appendix A. Example of DetNet data plane operation . . . . . . . 43 + Appendix B. Example of pinned paths using IPv6 . . . . . . . . . 44 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 44 1. Introduction Deterministic Networking (DetNet) is a service that can be offered by a network to DetNet flows. DetNet provides these flows extremely low packet loss rates and assured maximum end-to-end delivery latency. General background and concepts of DetNet can be found in [I-D.ietf-detnet-architecture]. This document specifies the DetNet data plane and the on-wire @@ -652,20 +652,62 @@ Furthermore, DetNet domain IP-header format may collide with IP- header format used by the source of a flow. Implementing such an approach requires that source encapsulation is in-line with DetNet domain encapsulation format, however we do not intend to mandate end- systems' encapsulation format (see former text: As a general rule, DetNet domains MUST be capable to forward any Data-flows and the DetNet domain MUST NOT intend to mandate end-system encapsulation format.). +5.2.2.3. Simplified IP Service + + In this case there is no "tunneling" below the DetNet Service, but + the DetNet Service flows are mapped to each link / sub net using its + technology specific methods. The DetNet IP header containes the IP + address destination DetNet end system. The data-flow IP header MUST + be preserved as-is. + + This solution provides end to end DetNet service consisting of + congestion protection and latency control and the rouse allocation + (queuing, policing, shaping) done using the underlying link / sub net + specific mechanisms. Compared to previously described DetNet routing + services, the service protections (packet replication and packet + emilination functions) and not provided end to end, but per + underlying layer-2 link / sub net. + + +------+ +------+ + | X | | X | + +======+ +------+ + End-system | IP | | IP | + -----+------+-------+======+--- --+======+-- + DetNet |L2/SbN| |L2/SbN| + +------+ +------+ + + Figure 9: Encapsulation of DetNet Routing in simplified IP service L3 + end-systems + + Note: the DetNet Service Flow MUST be mapped to the link / sub net + specific resources using an underlying system specific means. This + implies each DetNet aware node on path MUST look into the transported + DetNet Service Flow packet and utilize e.g., a five tuple to find out + the required mapping in a node. As noted earlier, the Service + Protection is done within each link / sub net independently using the + domain specific mechanisms (due the lack of a unified end to end + sequencing information that would be available for intermediate + nodes). If end to end service protection is desired that can be + implemented, for example, by the DetNet end systems using Layer-4 + transport protocols or application protocols. However, these are out + of scope of this document. + + [Editor's note: the service protection to be clarified further.] + 5.3. DetNet Inter-Working Function (DN-IWF) 5.3.1. Networks with multiple technology segments There are network scenarios, where the DetNet domain contains multiple technology segments (IP, MPLS) and all those segments are under the same administrative control. Furthermore, DetNet nodes may be interconnected via TSN segments. An important aspect of DetNet network design is placement of DetNet @@ -694,21 +736,21 @@ \_______/ \___/ +------------+ +------------------E----+ | | +----+ | | +----R---+ | +----+ |src |-------R +---+ | E----------+ dst| +----+ | | E--------+ +----+ +--------------R | +-----------------+ - Figure 9: Optimal replication and elimination placement across + Figure 10: Optimal replication and elimination placement across technology segments example 5.3.2. DN-IWF related considerations The ultimate goal of DN-IWF is to (1) match and (2) translate segment specific flow attributes. The DN-IWF ensures that segment specific attributes comprise per domain unique attributes for the whole DetNet domain. This characteristic can ensure that DetNet functions can be based on per domain attributes and not per segment attributes. @@ -732,37 +774,37 @@ of-scope for (2) TSN - IP, (3) TSN - MPLS. Note2: incompatible format of Seq.Number with TSN. Simplest implementation of DN-IWF is provided if the flow attributes have the same format. Such a common denominator of the tunnel encapsulation format is the pseudowire encapsulation over both IP and MPLS. Placeholder - Figure 10: FIGURE Placeholder PW over X + Figure 11: FIGURE Placeholder PW over X 6. MPLS-based DetNet data plane solution 6.1. DetNet specific packet fields The DetNet data plane encapsulation MUST include two DetNet specific information elements in each packet of a DetNet flow: (1) a flow identification and (2) a sequence number. The DetNet data plane encapsulation may consists further elements used for overlay tunneling, to distinguish between DetNet member flows of the same DetNet compound flow or to support OAM functions. 6.2. Data plane encapsulation - Figure 11 illustrates a DetNet data plane MPLS encapsulation. The + Figure 12 illustrates a DetNet data plane MPLS encapsulation. The MPLS-based encapsulation of the DetNet flows is a good fit for the Layer-2 interconnect deployment cases (see Figure 1). Furthermore, end to end DetNet service i.e., native DetNet deployment (see Figure 2) is also possible if DetNet end systems are capable of initiating and termination MPLS encapsulated packets. Transport of IP encapsulated DetNet flows, see Section 7, over MPLS-based DetNet data plane is also possible. Interworking between PW- and IPv6-based encapsulations is discussed further in Section 8.6. The MPLS-based DetNet data plane encapsulation consists of: @@ -800,46 +842,46 @@ +---------------------------------+ +--> DetNet data plane | S-Label | | MPLS encapsulation +---------------------------------+ <--/ | T-Label(s) | +---------------------------------+ | Data-Link | +---------------------------------+ | Physical | +---------------------------------+ - Figure 11: Encapsulation of a DetNet flow in an MPLS(-TP) PSN + Figure 12: Encapsulation of a DetNet flow in an MPLS(-TP) PSN 6.3. DetNet control word A DetNet control word (d-CW) conforms to the Generic PW MPLS Control - Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 12. + Word (PWMCW) defined in [RFC4385] and is illustrated in Figure 13. The upper nibble of the d-CW MUST be set to zero (0). The effective sequence number bit length is between 0 and 28 bits, and configured either by a control plane or manually for each DetNet flow. The sequence number is aligned to the right (least significant bits) and unused bits MUST be set to zero (0). Each DetNet flow MUST have its own sequence number counter. The sequence number is incremented by one for each new packet. The d-CW MUST always be present in a packet. In a case the sequence number is not used (e.g., for DetNet-t-flows) the control plane or the manual configuration has to define zero (0) bit length seqeunce number and the value of the sequence number MUST be set to zero (0). 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0| Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Figure 12: DetNet Control Word + Figure 13: DetNet Control Word 6.4. Flow identification DetNet flow identification at a DetNet service layer is realized by an S-label. It maps a Detnet flow to a specific d-CW in a DetNet node. The S-label used for flow identification MUST be bottom label of the label stack for a DetNet-s- or DetNet-st-flow and MUST precede the d-CW. An S-label for a single DetNet flow does not need to be unique DetNet @@ -898,21 +940,21 @@ Client AC | | Repl. | | | <---------->o o <-----X-> o o<----------> | | Elim. | | +-------------+ +-------------+ | | | | Client AC | | | | <---------->o o <-------> o o<----------> | | | | +---------------------------------------+ - Figure 13: DetNet Edge Node processing + Figure 14: DetNet Edge Node processing An edge node participates to the packet replication and duplication elimination. Required processing is done within an extended forwarder function. In the case the native service processing (NSP) is IEEE 802.1CB [IEEE8021CB] capable, the packet replication and duplicate elimination MAY entirely be done in the NSP and bypassing the DetNet flow encapsulation and logic entirely, and thus is able to operate over unmodified implementation and deployment. The NSP approach works only between edge nodes and cannot make use of relay nodes (see Section 6.5.2). @@ -1002,21 +1044,21 @@ | | Repl. | | | ----------->o o ------+-> o o-----------> | | | | Ingress +-------------+ +-------------+ | | | | Egress | | | | ----------->o o --------> o o-----------> | | | | +---------------------------------------+ - Figure 14: DetNet Relay Node processing + Figure 15: DetNet Relay Node processing Comment #35 SB> Somewhere in the dp document there needs to be a note of the requirement for interfaces to do fast exchange of counter state, and a note to those planning the network and designing the control plane that they need to provide support for this. Discussion: We kinf of agree but also think the above exchange or synchronization of counter states is not in our scope to solve. @@ -1031,21 +1073,21 @@ TBD. 6.6.2. Explicit routes TBD. 7. IPv6-based DetNet data plane solution 7.1. Data plane encapsulation - Figure 15 illustrates a DetNet native IPv6 encapsulation. The native + Figure 16 illustrates a DetNet native IPv6 encapsulation. The native IPv6 encapsulation is meant for end to end Detnet service use cases, where the end stations are DetNet-aware (see Figure 3). Technically it is possible to use the IPv6 encapsulation to tunnel any traffic over a DetNet enabled network, which would make native IPv6 encapsulation also a valid data plane choice for an interconnect use case (see Figure 1). The native IPv6-based DetNet data plane encapsulation consists of: o IPv6 header as the transport protocol. @@ -1092,24 +1134,24 @@ | DetNet Flow | | Payload | | | /---------------------------------\ H Optional DetNet DstOpt Hdr H \---------------------------------/ | IPv6 header | | (with set Flow label) | +---------------------------------+ - Figure 15: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow + Figure 16: Encapsulation of a native IPv6 DetNet-s- or DetNet-st-flow without a routing header - Figure 16 illustrates an IPv6 packet for the case where a routing + Figure 17 illustrates an IPv6 packet for the case where a routing header has been added into the packet by a DetNet-aware end system (again assuming DetNet-s- or DetNet-st-flows). Note that the use of routing header such as the one with the segment routing option is not mandatory for explicit routes. Similar functionality can be arranged using other means as well (e.g., using policy routing or layer-2 means). +---------------------------------+ | | | DetNet Flow | @@ -1119,52 +1161,52 @@ H DetNet DstOpt Hdr H \---------------------------------/ | Routing header | /---------------------------------\ H DetNet DstOpt Hdr H \---------------------------------/ | IPv6 header | | (with set Flow label) | +---------------------------------+ - Figure 16: Encapsulation of a native IPv6 DetNet-s- or DetNet-st- + Figure 17: Encapsulation of a native IPv6 DetNet-s- or DetNet-st- flows with routing header IPv6 extension headers can only be inserted by a node that initiated the IPv6 packet. IPv6 extension headers, except for the Hop-by-Hop Option headers, can only be processed by an IPv6 node that is identified by the Destination Address field of the IPv6 header (see Section 0 of [RFC8200]. Therefore, if a DetNet-aware end system only inserted the DetNet Destination Option into the IPv6 but e.g., a DetNet Edge node is configured to enforce an explicit route for the IPv6 packet using a source routing header, then it has no other possibility than add an outer tunneling IPv6 header with required extension headers in it. The processing of IPv6 packets in a DetNet Edge node is discussed further in Section 7.4.1. 7.2. DetNet destination option A DetNet flow must carry sequencing information for packet replication and elimination function (PREF) purposes. This document specifies a new IPv6 Destination Option: the DetNet Destination Option for that purpose. The format of the option is illustrated in - Figure 17. + Figure 18. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TBD1 | 4 | 0 | 28 bit sequence +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - Figure 17: DetNet Destination Option + Figure 18: DetNet Destination Option The Option Type for the DetNet Destination Option is set to TBD1. [To be removed from the final version of the document: The Option Type MUST have the two most significant bits set to 10b] If an IPv6 packet gets dropped due the DetNet Service layer processing based on the DetNet Destination Option an ICMPv6 packet of any type MUST NOT be sent back to the source of the packet. 7.3. Flow identification @@ -1244,21 +1286,21 @@ at DetNet Edge nodes. Possible explicit routes between edge nodes are arranged by other than IPv6 specific means. 4. A DetNet-s- or a DetNet-st-flow packet arrives at an ingress DetNet Edge node and multiple DetNet Relay nodes may process DetNet flow packets before reaching an egress DetNet Edge node. Explicit routes between edge nodes has to be arranged by IPv6 specific means. A generic DetNet IPv6 encapsulation for a DetNet flow packet between - DetNet Edge nodes is shown in Figure 18. Essentially every time an + DetNet Edge nodes is shown in Figure 19. Essentially every time an igress DetNet Edge node has to insert something into the DetNet flow packet it has to add an outer tunneling IPv6 header, which then contain possible additional extension headers. +---------------------------------+ | | | DetNet Flow | | Payload | | | +---------------------------------+ @@ -1268,21 +1310,21 @@ | (with set Flow label) (1) | +---------------------------------+ <--+ | Optional Routing header | | +---------------------------------+ | | Optional DetNet DstOpt Hdr (2) | +-- Added/removed by +---------------------------------+ | DetNet Edge node | Outer IPv6 header | | | (with set Flow label) (2) | | +---------------------------------+ <--+ - Figure 18: Encapsulation of a DetNet-flow IPv6 packet at the DetNet + Figure 19: Encapsulation of a DetNet-flow IPv6 packet at the DetNet Edge node 7.4.1.1. Ingress DetNet Edge node processing Case 1) MAY require an addition of the DetNet Destination Option if packet reordering is requested at the egress DetNet Edge node. Otherwise, no modifications except rewriting the IPv6 header flow label to the packet is done. If modifications are required then: o The outer IPv6 header is added with the Source Address set to the