draft-ietf-detnet-data-plane-framework-04.txt | draft-ietf-detnet-data-plane-framework-05.txt | |||
---|---|---|---|---|
DetNet B. Varga, Ed. | DetNet B. Varga, Ed. | |||
Internet-Draft J. Farkas | Internet-Draft J. Farkas | |||
Intended status: Informational Ericsson | Intended status: Informational Ericsson | |||
Expires: August 6, 2020 L. Berger | Expires: October 25, 2020 L. Berger | |||
LabN Consulting, L.L.C. | LabN Consulting, L.L.C. | |||
A. Malis | A. Malis | |||
Independent | Malis Consulting | |||
S. Bryant | S. Bryant | |||
Futurewei Technologies | Futurewei Technologies | |||
February 3, 2020 | April 23, 2020 | |||
DetNet Data Plane Framework | DetNet Data Plane Framework | |||
draft-ietf-detnet-data-plane-framework-04 | draft-ietf-detnet-data-plane-framework-05 | |||
Abstract | Abstract | |||
This document provides an overall framework for the DetNet data | This document provides an overall framework for the DetNet data | |||
plane. It covers concepts and considerations that are generally | plane. It covers concepts and considerations that are generally | |||
common to any Deterministic Networking data plane specification. | common to any Deterministic Networking data plane specification. | |||
Status of This Memo | Status of This Memo | |||
This Internet-Draft is submitted in full conformance with the | This Internet-Draft is submitted in full conformance with the | |||
skipping to change at page 1, line 38 ¶ | skipping to change at page 1, line 38 ¶ | |||
Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
Drafts is at https://datatracker.ietf.org/drafts/current/. | Drafts is at https://datatracker.ietf.org/drafts/current/. | |||
Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
This Internet-Draft will expire on August 6, 2020. | This Internet-Draft will expire on October 25, 2020. | |||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2020 IETF Trust and the persons identified as the | Copyright (c) 2020 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
(https://trustee.ietf.org/license-info) in effect on the date of | (https://trustee.ietf.org/license-info) in effect on the date of | |||
publication of this document. Please review these documents | publication of this document. Please review these documents | |||
skipping to change at page 2, line 18 ¶ | skipping to change at page 2, line 18 ¶ | |||
Table of Contents | Table of Contents | |||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 | 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 | |||
2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 | 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 | |||
2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 | 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 | |||
3. DetNet Data Plane Overview . . . . . . . . . . . . . . . . . 4 | 3. DetNet Data Plane Overview . . . . . . . . . . . . . . . . . 4 | |||
3.1. Data Plane Characteristics . . . . . . . . . . . . . . . 6 | 3.1. Data Plane Characteristics . . . . . . . . . . . . . . . 6 | |||
3.1.1. Data Plane Technology . . . . . . . . . . . . . . . . 6 | 3.1.1. Data Plane Technology . . . . . . . . . . . . . . . . 6 | |||
3.1.2. Data Plane Format . . . . . . . . . . . . . . . . . . 6 | 3.1.2. Encapsulation . . . . . . . . . . . . . . . . . . . . 6 | |||
3.2. Encapsulation . . . . . . . . . . . . . . . . . . . . . . 6 | 3.2. DetNet-specific Metadata . . . . . . . . . . . . . . . . 7 | |||
3.3. DetNet Specific Metadata . . . . . . . . . . . . . . . . 7 | 3.3. DetNet IP Data Plane . . . . . . . . . . . . . . . . . . 8 | |||
3.4. DetNet IP Data Plane . . . . . . . . . . . . . . . . . . 8 | 3.4. DetNet MPLS Data Plane . . . . . . . . . . . . . . . . . 8 | |||
3.5. DetNet MPLS Data Plane . . . . . . . . . . . . . . . . . 9 | 3.5. Further DetNet Data Plane Considerations . . . . . . . . 9 | |||
3.6. Further DetNet Data Plane Considerations . . . . . . . . 9 | 3.5.1. Per Flow Related Functions . . . . . . . . . . . . . 9 | |||
3.6.1. Per Flow Related Functions . . . . . . . . . . . . . 9 | 3.5.2. Service Protection . . . . . . . . . . . . . . . . . 11 | |||
3.6.2. Service Protection . . . . . . . . . . . . . . . . . 11 | 3.5.3. Aggregation Considerations . . . . . . . . . . . . . 13 | |||
3.6.3. Aggregation Considerations . . . . . . . . . . . . . 13 | 3.5.4. End-System-Specific Considerations . . . . . . . . . 14 | |||
3.6.4. End-System-Specific Considerations . . . . . . . . . 14 | 3.5.5. Sub-Network Considerations . . . . . . . . . . . . . 15 | |||
3.6.5. Sub-Network Considerations . . . . . . . . . . . . . 15 | ||||
4. Controller Plane (Management and Control) | 4. Controller Plane (Management and Control) | |||
Considerations . . . . . . . . . . . . . . . . . . . . . . . 16 | Considerations . . . . . . . . . . . . . . . . . . . . . . . 15 | |||
4.1. DetNet Controller Plane Requirements . . . . . . . . . . 16 | 4.1. DetNet Controller Plane Requirements . . . . . . . . . . 15 | |||
4.2. Generic Controller Plane Considerations . . . . . . . . . 17 | 4.2. Generic Controller Plane Considerations . . . . . . . . . 17 | |||
4.2.1. Flow Aggregation Control . . . . . . . . . . . . . . 18 | 4.2.1. Flow Aggregation Control . . . . . . . . . . . . . . 17 | |||
4.2.2. Explicit Routes . . . . . . . . . . . . . . . . . . . 19 | 4.2.2. Explicit Routes . . . . . . . . . . . . . . . . . . . 18 | |||
4.2.3. Contention Loss and Jitter Reduction . . . . . . . . 19 | 4.2.3. Contention Loss and Jitter Reduction . . . . . . . . 19 | |||
4.2.4. Bidirectional Traffic . . . . . . . . . . . . . . . . 20 | 4.2.4. Bidirectional Traffic . . . . . . . . . . . . . . . . 19 | |||
4.3. Packet Replication, Elimination, and Ordering (PREOF) . . 21 | 4.3. Packet Replication, Elimination, and Ordering (PREOF) . . 20 | |||
5. Security Considerations . . . . . . . . . . . . . . . . . . . 21 | 5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 | |||
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 | 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 | |||
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 | 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 | |||
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 | 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 | |||
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 | 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 | |||
9.1. Normative References . . . . . . . . . . . . . . . . . . 22 | 9.1. Normative References . . . . . . . . . . . . . . . . . . 22 | |||
9.2. Informative References . . . . . . . . . . . . . . . . . 23 | 9.2. Informative References . . . . . . . . . . . . . . . . . 22 | |||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 | Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 | |||
1. Introduction | 1. Introduction | |||
DetNet (Deterministic Networking) provides a capability to carry | DetNet (Deterministic Networking) provides a capability to carry | |||
specified unicast or multicast data flows for real-time applications | specified unicast or multicast data flows for real-time applications | |||
with extremely low packet loss rates and assured maximum end-to-end | with extremely low packet loss rates and assured maximum end-to-end | |||
delivery latency. A description of the general background and | delivery latency. A description of the general background and | |||
concepts of DetNet can be found in [RFC8655]. | concepts of DetNet can be found in [RFC8655]. | |||
This document describes the concepts needed by any DetNet data plane | This document describes the concepts needed by any DetNet data plane | |||
specification and provides considerations for any corresponding | specification (i.e., the DetNet-specific use of packet header fields) | |||
implementation. It covers the building blocks that provide the | and provides considerations for any corresponding implementation. It | |||
DetNet service, the DetNet service sub-layer and the DetNet | covers the building blocks that provide the DetNet service, the | |||
forwarding sub-layer functions as described in the DetNet | DetNet service sub-layer and the DetNet forwarding sub-layer | |||
Architecture. | functions as described in the DetNet Architecture. | |||
The DetNet Architecture models the DetNet related data plane | The DetNet Architecture models the DetNet related data plane | |||
functions decomposed into two sub-layers: a service sub-layer and a | functions decomposed into two sub-layers: a service sub-layer and a | |||
forwarding sub-layer. The service sub-layer is used to provide | forwarding sub-layer. The service sub-layer is used to provide | |||
DetNet service protection and reordering. The forwarding sub-layer | DetNet service protection and reordering. The forwarding sub-layer | |||
leverages Traffic Engineering mechanisms and provides congestion | leverages Traffic Engineering mechanisms and provides congestion | |||
protection (low loss, assured latency, and limited out-of-order | protection (low loss, assured latency, and limited out-of-order | |||
delivery). A particular forwarding sub-layer may have capabilities | delivery). A particular forwarding sub-layer may have capabilities | |||
that are not available on other forwarding-sub layers. DetNet makes | that are not available on other forwarding-sub layers. DetNet makes | |||
use of the existing forwarding sub-layers with their respective | use of the existing forwarding sub-layers with their respective | |||
capabilities and does not require 1:1 equivalence between different | capabilities and does not require 1:1 equivalence between different | |||
forwarding sub-layer capabilities. | forwarding sub-layer capabilities. | |||
As part of the service sub-layer functions, this document describes | As part of the service sub-layer functions, this document describes | |||
typical DetNet node data plane operation. It describes the function | typical DetNet node data plane operation. It describes the | |||
and operation of the Packet Replication (PRF) Packet Elimination | functionality and operation of the Packet Replication (PRF), Packet | |||
(PEF) and the Packet Ordering (POF) functions within the service sub- | Elimination (PEF), and the Packet Ordering (POF) functions within the | |||
layer. Furthermore, it also describes the forwarding sub-layer. | service sub-layer. Furthermore, it describes the forwarding sub- | |||
layer. | ||||
DetNet flows may be carried over network technologies that can | As defined in [RFC8655], DetNet flows may be carried over network | |||
provide the DetNet required service characteristics. For example, | technologies that can provide the DetNet required service | |||
DetNet MPLS flows can be carried over IEEE 802.1 Time Sensitive | characteristics. For example, DetNet MPLS flows can be carried over | |||
Network (TSN) [IEEE802.1TSNTG] sub-networks. However, IEEE 802.1 TSN | IEEE 802.1 Time Sensitive Network (TSN) [IEEE802.1TSNTG] sub- | |||
support is not required in DetNet. TSN frame preemption is an | networks. However, IEEE 802.1 TSN support is not required in DetNet. | |||
example of a forwarding layer capability that is typically not | TSN frame preemption is an example of a forwarding layer capability | |||
replicated in other forwarding technologies. Most of DetNet benefits | that is typically not replicated in other forwarding technologies. | |||
can be gained by running over a data link layer that has not been | Most of DetNet benefits can be gained by running over a data link | |||
specifically enhanced to support all TSN capabilities but for certain | layer that has not been specifically enhanced to support all TSN | |||
networks and traffic mixes delay and jitter performance may vary due | capabilities but for certain networks and traffic mixes delay and | |||
to the forwarding sub-layer intrinsic properties. | jitter performance may vary due to the forwarding sub-layer intrinsic | |||
properties. | ||||
Different application flows (e.g., Ethernet, IP, etc.), can be mapped | Different application flows, such as Ethernet or IP, can be mapped on | |||
on top of DetNet. DetNet can optionally reuse header information | top of DetNet. DetNet can optionally reuse header information | |||
provided by, or shared with, applications. An example of shared | provided by, or shared with, applications. An example of shared | |||
header fields can be found in [I-D.ietf-detnet-ip]. | header fields can be found in [I-D.ietf-detnet-ip]. | |||
This document also covers basic concepts related to the controller | This document also covers basic concepts related to the controller | |||
plane and Operations, Administration, and Maintenance (OAM). Data | plane and Operations, Administration, and Maintenance (OAM). Data | |||
plane OAM specifics are out of scope for this document. | plane OAM specifics are out of scope for this document. | |||
2. Terminology | 2. Terminology | |||
2.1. Terms Used in This Document | 2.1. Terms Used in This Document | |||
This document uses the terminology established in the DetNet | This document uses the terminology established in the DetNet | |||
architecture [RFC8655], and the reader is assumed to be familiar with | architecture [RFC8655], and the reader is assumed to be familiar with | |||
that document and its terminology. | that document and its terminology. | |||
2.2. Abbreviations | 2.2. Abbreviations | |||
The following abbreviations are used in this document: | The following abbreviations are used in this document: | |||
BGP Border Gateway Protocol. | BGP Border Gateway Protocol. | |||
CW Control Word. | ||||
d-CW DetNet Control Word. | d-CW DetNet Control Word. | |||
DetNet Deterministic Networking. | DetNet Deterministic Networking. | |||
DN DetNet. | DN DetNet. | |||
GMPLS Generalized Multiprotocol Label Switching. | GMPLS Generalized Multiprotocol Label Switching. | |||
GRE Generic Routing Encapsulation. | GRE Generic Routing Encapsulation. | |||
IPSec IP Security. | IPSec IP Security. | |||
L2 Layer 2. | L2 Layer 2. | |||
LSP Label Switched Path. | LSP Label Switched Path. | |||
LSR Label Switching Router. | ||||
MPLS Multiprotocol Label Switching. | MPLS Multiprotocol Label Switching. | |||
MPLS-TE Multiprotocol Label Switching - Traffic Engineering. | ||||
OAM Operations, Administration, and Maintenance. | OAM Operations, Administration, and Maintenance. | |||
PCEP Path Computation Element Communication Protocol. | PCEP Path Computation Element Communication Protocol. | |||
PEF Packet Elimination Function. | PEF Packet Elimination Function. | |||
PRF Packet Replication Function. | PRF Packet Replication Function. | |||
PREOF Packet Replication, Elimination and Ordering Functions. | PREOF Packet Replication, Elimination and Ordering Functions. | |||
POF Packet Ordering Function. | POF Packet Ordering Function. | |||
PSN Packet Switched Network. | PSN Packet Switched Network. | |||
PW PseudoWire. | ||||
QoS Quality of Service. | QoS Quality of Service. | |||
S-Label DetNet "service" label. | S-Label DetNet "service" label. | |||
TDM Time-Division Multiplexing. | TDM Time-Division Multiplexing. | |||
TSN Time-Sensitive Network. | TSN Time-Sensitive Network. | |||
YANG Yet Another Next Generation. | YANG Yet Another Next Generation. | |||
3. DetNet Data Plane Overview | 3. DetNet Data Plane Overview | |||
This document describes how application flows, or app-flows, are | This document describes how application flows, or app-flows, are | |||
carried over DetNet networks. The DetNet Architecture, [RFC8655], | carried over DetNet networks. The DetNet Architecture [RFC8655] | |||
models the DetNet related data plane functions as decomposed into two | models the DetNet-related data plane functions as decomposed into two | |||
sub-layers: a service sub-layer and a forwarding sub-layer. | sub-layers: a service sub-layer and a forwarding sub-layer. | |||
Figure 1 reproduced from the [RFC8655],shows a logical DetNet service | Figure 1, reproduced from [RFC8655], shows a logical DetNet service | |||
with the two sub-layers. | with the two sub-layers. | |||
| packets going | ^ packets coming ^ | | packets going | ^ packets coming ^ | |||
v down the stack v | up the stack | | v down the stack v | up the stack | | |||
+-----------------------+ +-----------------------+ | +-----------------------+ +-----------------------+ | |||
| Source | | Destination | | | Source | | Destination | | |||
+-----------------------+ +-----------------------+ | +-----------------------+ +-----------------------+ | |||
| Service sub-layer: | | Service sub-layer: | | | Service sub-layer: | | Service sub-layer: | | |||
| Packet sequencing | | Duplicate elimination | | | Packet sequencing | | Duplicate elimination | | |||
| Flow replication | | Flow merging | | | Flow replication | | Flow merging | | |||
skipping to change at page 5, line 45 ¶ | skipping to change at page 5, line 45 ¶ | |||
techniques and traffic engineering methods, or it may do this through | techniques and traffic engineering methods, or it may do this through | |||
the assistance of its underlying connectivity. For example it may | the assistance of its underlying connectivity. For example it may | |||
call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN | call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN | |||
[IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for | [IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for | |||
packet queuing, as well as reservation and allocation of bandwidth | packet queuing, as well as reservation and allocation of bandwidth | |||
capacity resources. | capacity resources. | |||
The service sub-layer provides additional support beyond the | The service sub-layer provides additional support beyond the | |||
connectivity function of the forwarding sub-layer. An example of | connectivity function of the forwarding sub-layer. An example of | |||
this is Packet Replication, Elimination, and Ordering functions see | this is Packet Replication, Elimination, and Ordering functions see | |||
Section 4.3. The ordering (POF) uses sequence numbers added to | Section 4.3. The ordering function (POF) uses sequence numbers added | |||
packets enabling a range of packet order protection from simple | to packets enabling a range of packet order protection from simple | |||
ordering and dropping out-of-order packets to more complex reordering | ordering and dropping out-of-order packets to more complex reordering | |||
of a fixed number of out-of-order, minimally delayed packets. | of a fixed number of out-of-order, minimally delayed packets. | |||
Reordering requires buffer resources and has impact on the delay and | Reordering requires buffer resources and has impact on the delay and | |||
jitter of packets in the DetNet flow. | jitter of packets in the DetNet flow. | |||
The method of instantiating each of the layers is specific to the | The method of instantiating each of the layers is specific to the | |||
particular DetNet data plane method, and more than one approach may | particular DetNet data plane method, and more than one approach may | |||
be applicable to a given bearer network type. | be applicable to a given bearer network type. | |||
3.1. Data Plane Characteristics | 3.1. Data Plane Characteristics | |||
skipping to change at page 6, line 23 ¶ | skipping to change at page 6, line 23 ¶ | |||
3.1.1. Data Plane Technology | 3.1.1. Data Plane Technology | |||
The DetNet data plane is provided by the DetNet service and | The DetNet data plane is provided by the DetNet service and | |||
forwarding sub layers. The DetNet service sub-layer generally | forwarding sub layers. The DetNet service sub-layer generally | |||
provides its functions for the DetNet application flows by using or | provides its functions for the DetNet application flows by using or | |||
applying existing standardized headers and/or encapsulations. The | applying existing standardized headers and/or encapsulations. The | |||
Detnet forwarding sub-layer may provide capabilities leveraging that | Detnet forwarding sub-layer may provide capabilities leveraging that | |||
same header or encapsulation technology (e.g., DN IP or DN MPLS) or | same header or encapsulation technology (e.g., DN IP or DN MPLS) or | |||
it may be achieved by other technologies (e.g., Figure 2). DetNet is | it may be achieved by other technologies (e.g., Figure 2). DetNet is | |||
currently defined for operation over packet switched (IP) networks or | currently defined for operation over packet-switched (IP) networks or | |||
label switched (MPLS) networks. | label-switched (MPLS) networks. | |||
3.1.2. Data Plane Format | 3.1.2. Encapsulation | |||
DetNet encodes specific flow attributes (flow identity and sequence | DetNet encodes specific flow attributes (flow identity and sequence | |||
number) in packets. For example, in DetNet IP, zero encapsulation is | number) in packets. For example, in DetNet IP, zero encapsulation is | |||
used and no sequence number is available, and in DetNet MPLS, DetNet | used and no sequence number is available, and in DetNet MPLS, DetNet- | |||
specific information may be added explicitly to the packets in the | specific information may be added explicitly to the packets in the | |||
format of S-label and d-CW [I-D.ietf-detnet-mpls] . | format of S-label and d-CW [I-D.ietf-detnet-mpls] . | |||
3.2. Encapsulation | ||||
The encapsulation of a DetNet flow allows it to be sent over a data | The encapsulation of a DetNet flow allows it to be sent over a data | |||
plane technology other than its native type. DetNet uses header | plane technology other than its native type. DetNet uses header | |||
information to perform traffic classification, i.e., identify DetNet | information to perform traffic classification, i.e., identify DetNet | |||
flows, and provide DetNet service and forwarding functions. As | flows, and provide DetNet service and forwarding functions. As | |||
mentioned above, DetNet may add headers, as is the case for DN MPLS, | mentioned above, DetNet may add headers, as is the case for DN MPLS, | |||
or may use headers that are already present, as is the case in DN IP. | or may use headers that are already present, as is the case in DN IP. | |||
Figure 2 illustrates some relationships between the components. | Figure 2 illustrates some relationships between the components. | |||
+-----+ | +-----+ | |||
| TSN | | | TSN | | |||
skipping to change at page 7, line 25 ¶ | skipping to change at page 7, line 15 ¶ | |||
The use of encapsulation is also required if additional information | The use of encapsulation is also required if additional information | |||
(metadata) is needed by the DetNet data plane and there is either no | (metadata) is needed by the DetNet data plane and there is either no | |||
ability to include it in the client data packet, or the specification | ability to include it in the client data packet, or the specification | |||
of the client data plane does not permit the modification of the | of the client data plane does not permit the modification of the | |||
packet to include additional data. An example of such metadata is | packet to include additional data. An example of such metadata is | |||
the inclusion of a sequence number required by the PREOF function. | the inclusion of a sequence number required by the PREOF function. | |||
Encapsulation may also be used to carry or aggregate flows for | Encapsulation may also be used to carry or aggregate flows for | |||
equipment with limited DetNet capability. | equipment with limited DetNet capability. | |||
3.3. DetNet Specific Metadata | 3.2. DetNet-specific Metadata | |||
The DetNet data plane can provide or carry metadata: | The DetNet data plane can provide or carry the following metadata: | |||
1. Flow-ID | 1. Flow-ID | |||
2. Sequence Number | 2. Sequence Number | |||
The DetNet data plane framework supports a Flow-ID (for | The DetNet data plane framework supports a Flow-ID (for | |||
identification of the flow or aggregate flow) and/or a Sequence | identification of the flow or aggregate flow) and/or a Sequence | |||
Number (for PREOF) for each DetNet flow. The DetNet Service sub- | Number (for PREOF) for each DetNet flow. The DetNet Service sub- | |||
layer requires both; the DetNet forwarding sub-layer requires only | layer requires both; the DetNet forwarding sub-layer requires only | |||
Flow-ID. Metadata can also be used for OAM indications and | Flow-ID. Metadata can also be used for OAM indications and | |||
instrumentation of DetNet data plane operation. | instrumentation of DetNet data plane operation. | |||
Metadata can be included implicit or explicit. Explicit means that a | Metadata inclusion can be implicit or explicit. Explicit inclusions | |||
dedicated header field is used to include metadata in a DetNet | involve a dedicated header field that is used to include metadata in | |||
packet. In case of implicit method a part of an already existing | a DetNet packet. In the implicit method, part of an already existing | |||
header field is used to encode the metadata. | header field is used to encode the metadata. | |||
Explicit inclusion of metadata is possible through the use of IP | Explicit inclusion of metadata is possible through the use of IP | |||
options or IP extension headers. New IP options are almost | options or IP extension headers. New IP options are almost | |||
impossible to get standardized or to deploy in an operational network | impossible to get standardized or to deploy in an operational network | |||
and will not be discussed further in this text. IPv6 extensions | and will not be discussed further in this text. IPv6 extensions | |||
headers are finding popularity in current IPv6 development work, | headers are finding popularity in current IPv6 development work, | |||
particularly in connection with Segment Routing of IPv6 (SRv6) and IP | particularly in connection with Segment Routing of IPv6 (SRv6) and IP | |||
OAM. The design of a new IPv6 extension header or the modification | OAM. The design of a new IPv6 extension header or the modification | |||
of an existing one is a technique available in the tool box of the | of an existing one is a technique available in the tool box of the | |||
DetNet IP data plane designer. | DetNet IP data plane designer. | |||
Explicit inclusion of metadata in an IP packet is also possible | Explicit inclusion of metadata in an IP packet is also possible | |||
through the inclusion of an MPLS label stack and the MPLS DetNet | through the inclusion of an MPLS label stack and the MPLS DetNet | |||
Control Word using one of the methods for carrying MPLS over IP | Control Word using one of the methods for carrying MPLS over IP | |||
[I-D.ietf-detnet-mpls-over-udp-ip]. This is described in more detail | [I-D.ietf-detnet-mpls-over-udp-ip]. This is described in more detail | |||
in Section 3.6.5. | in Section 3.5.5. | |||
Implicit metadata in IP can be included through the use of the | Implicit metadata in IP can be included through the use of the | |||
network programming paradigm | network programming paradigm | |||
[I-D.ietf-spring-srv6-network-programming] in which the suffix of an | [I-D.ietf-spring-srv6-network-programming] in which the suffix of an | |||
IPv6 address is used to encode additional information for use by the | IPv6 address is used to encode additional information for use by the | |||
network of the receiving host. | network of the receiving host. | |||
Some MPLS examples of implicit metadata include the sequence number | Some MPLS examples of implicit metadata include the sequence number | |||
for use by the PREOF function, or even all the essential information | for use by the PREOF function, or even all the essential information | |||
being included into the DetNet over MPLS label stack (the DetNet | being included into the DetNet over MPLS label stack (the DetNet | |||
Control Word and the DetNet Service label). | Control Word and the DetNet Service label). | |||
3.4. DetNet IP Data Plane | 3.3. DetNet IP Data Plane | |||
An IP data plane may operate natively or through the use of an | An IP data plane may operate natively or through the use of an | |||
encapsulation. Many types of IP encapsulation can satisfy DetNet | encapsulation. Many types of IP encapsulation can satisfy DetNet | |||
requirements and it is anticipated that more than one encapsulation | requirements and it is anticipated that more than one encapsulation | |||
may be deployed, for example GRE, IPSec etc. | may be deployed, for example GRE, IPSec. | |||
One method of operating an IP DetNet data plane without encapsulation | One method of operating an IP DetNet data plane without encapsulation | |||
is to use "6-tuple" based flow identification, where "6-tuple" refers | is to use "6-tuple" based flow identification, where "6-tuple" refers | |||
to information carried in IP and higher layer protocol headers. | to information carried in IP and higher layer protocol headers. | |||
General background on the use of IP headers, and "6-tuples", to | General background on the use of IP headers, and "6-tuples", to | |||
identify flows and support Quality of Service (QoS) can be found in | identify flows and support Quality of Service (QoS) can be found in | |||
[RFC3670]. [RFC7657] provides useful background on differentiated | [RFC3670]. [RFC7657] provides useful background on differentiated | |||
services (DiffServ) and "tuple" based flow identification. DetNet | services (DiffServ) and "tuple" based flow identification. DetNet | |||
flow aggregation may be enabled via the use of wildcards, masks, | flow aggregation may be enabled via the use of wildcards, masks, | |||
prefixes and ranges. The operation of this method is described in | prefixes and ranges. The operation of this method is described in | |||
detail in [I-D.ietf-detnet-ip]. | detail in [I-D.ietf-detnet-ip]. | |||
The DetNet forwarding plane may use explicit route capabilities and | The DetNet forwarding plane may use explicit route capabilities and | |||
traffic engineering capabilities to provide a forwarding sub-layer | traffic engineering capabilities to provide a forwarding sub-layer | |||
that is responsible for providing resource allocation and explicit | that is responsible for providing resource allocation and explicit | |||
routes. It is possible to include such information in a native IP | routes. It is possible to include such information in a native IP | |||
packet explicitly, or implicitly. | packet explicitly, or implicitly. | |||
3.5. DetNet MPLS Data Plane | 3.4. DetNet MPLS Data Plane | |||
MPLS provides a forwarding sub-layer for traffic over implicit and | MPLS provides a forwarding sub-layer for traffic over implicit and | |||
explicit paths to the point in the network where the next DetNet | explicit paths to the point in the network where the next DetNet | |||
service sub-layer action needs to take place. It does this through | service sub-layer action needs to take place. It does this through | |||
the use of a stack of one or more labels with various forwarding | the use of a stack of one or more labels with various forwarding | |||
semantics. | semantics. | |||
MPLS also provides the ability to identify a service instance that is | MPLS also provides the ability to identify a service instance that is | |||
used to process the packet through the use of a label that maps the | used to process the packet through the use of a label that maps the | |||
packet to a service instance. | packet to a service instance. | |||
In cases where metadata is needed to process an MPLS encapsulated | In cases where metadata is needed to process an MPLS encapsulated | |||
packet at the service sub-layer, the d-CW [I-D.ietf-detnet-mpls], | packet at the service sub-layer, the d-CW [I-D.ietf-detnet-mpls], | |||
[RFC4385],can be used. Although such d-CWs are frequently 32 bits | [RFC4385] can be used. Although such d-CWs are frequently 32 bits | |||
long, there is no architectural constraint on its size of this | long, there is no architectural constraint on the size of this | |||
structure, only the requirement that it is fully understood by all | structure, only the requirement that it is fully understood by all | |||
parties operating on it in the DetNet service sub-layer. The | parties operating on it in the DetNet service sub-layer. The | |||
operation of this method is described in detail in | operation of this method is described in detail in | |||
[I-D.ietf-detnet-mpls]. | [I-D.ietf-detnet-mpls]. | |||
3.6. Further DetNet Data Plane Considerations | 3.5. Further DetNet Data Plane Considerations | |||
This section provides informative considerations related to providing | This section provides informative considerations related to providing | |||
DetNet service to flows which are identified based on their header | DetNet service to flows which are identified based on their header | |||
information. | information. | |||
3.6.1. Per Flow Related Functions | 3.5.1. Per Flow Related Functions | |||
At a high level, the following functions are provided on a per flow | At a high level, the following functions are provided on a per flow | |||
basis. | basis. | |||
3.6.1.1. Reservation and Allocation of resources | 3.5.1.1. Reservation and Allocation of resources | |||
Reservation of resources can allocate resources to specific DetNet | Resources might be reserved in order to make them available for | |||
flows. This can eliminate packet contention and packet loss for | allocation to specific DetNet flows. This can eliminate packet | |||
DetNet traffic. This also can reduce jitter for DetNet traffic. | contention and packet loss for DetNet traffic. This also can reduce | |||
Resources allocated to a DetNet flow protect it from other traffic | jitter for DetNet traffic. Resources allocated to a DetNet flow | |||
flows. On the other hand, DetNet flows are assumed to behave with | protect it from other traffic flows. On the other hand, DetNet flows | |||
respect to the reserved traffic profile. Misbehaving DetNet flows | are assumed to behave with respect to the reserved traffic profile. | |||
must be able to be detected and ensure that they do not compromise | It must be possible to detect misbehaving DetNet flows and to ensure | |||
QoS of other flows. Queuing, policing, and shaping policies can be | that they do not compromise QoS of other flows. Queuing, policing, | |||
used to ensure that the allocation of resources reserved for DetNet | and shaping policies can be used to ensure that the allocation of | |||
is met. | resources reserved for DetNet is met. | |||
3.6.1.2. Explicit routes | 3.5.1.2. Explicit routes | |||
Use of a specific path for a flow. This allows control of the | A flow can be routed over a specific, pre-computed path. This allows | |||
network delay by steering the packet with the ability to influence | control of the network delay by steering the packet with the ability | |||
the physical path. Explicit routes complement reservation by | to influence the physical path. Explicit routes complement | |||
ensuring that a consistent path can be associated with its resources | reservation by ensuring that a consistent path can be associated with | |||
for the duration of that path. Coupled with the traffic mechanism, | its resources for the duration of that path. Coupled with the | |||
this limits misordering and bounds latency. Explicit route | traffic mechanism, this limits misordering and bounds latency. | |||
computation can encompass a wide set of constraints and optimize the | Explicit route computation can encompass a wide set of constraints | |||
path for a certain characteristic e.g. highest bandwidth or lowest | and can optimize the path for a certain characteristic, e.g., highest | |||
jitter. In these cases the "best" path for any set of | bandwidth or lowest jitter. In these cases the "best" path for any | |||
characteristics may not be a shortest path. The selection of path | set of characteristics may not be a shortest path. The selection of | |||
can take into account multiple network metrics. Some of these | path can take into account multiple network metrics. Some of these | |||
metrics are measured and distributed by the routing system as traffic | metrics are measured and distributed by the routing system as traffic | |||
engineering metrics. | engineering metrics. | |||
3.6.1.3. Service protection | 3.5.1.3. Service protection | |||
Use of multiple packet streams using multiple paths, for example 1+1 | Service protection involves use of multiple packet streams using | |||
or 1:1 linear protection. For DetNet this primarily relates to | multiple paths, for example 1+1 or 1:1 linear protection. For | |||
packet replication and elimination capabilities. MPLS offers a | DetNet, this primarily relates to packet replication and elimination | |||
number of protection schemes. MPLS hitless protection can be used to | capabilities. MPLS offers a number of protection schemes. MPLS | |||
switch traffic to an already established path in order to restore | hitless protection can be used to switch traffic to an already | |||
delivery rapidly after a failure. Path changes, even in the case of | established path in order to restore delivery rapidly after a | |||
failure recovery, can lead to the out of order delivery of data | failure. Path changes, even in the case of failure recovery, can | |||
requiring packet ordering functions either within the DetNet service | lead to the out of order delivery of data requiring packet ordering | |||
or at a high layer in the application traffic. Establishment of new | functions either within the DetNet service or at a high layer in the | |||
paths after a failure is out of scope for DetNet services. | application traffic. Establishment of new paths after a failure is | |||
out of scope for DetNet services. | ||||
3.6.1.4. Network Coding | 3.5.1.4. Network Coding | |||
Network Coding, [nwcrg] not to be confused with network programming, | Network Coding [nwcrg], not to be confused with network programming, | |||
comprises several techniques where multiple data flows are encoded. | comprises several techniques where multiple data flows are encoded. | |||
These resulting flows can then be sent on different paths. The | These resulting flows can then be sent on different paths. The | |||
encoding operation can combine flows and error recovery information. | encoding operation can combine flows and error recovery information. | |||
When the encoded flows are decoded and recombined the original flows | When the encoded flows are decoded and recombined the original flows | |||
can be recovered. Note that Network coding uses an alternative to | can be recovered. Note that Network coding uses an alternative to | |||
packet by packet PREOF. Therefore, for certain network topologies | packet by packet PREOF. Therefore, for certain network topologies | |||
and traffic loads, Network Coding can be used to improve a network's | and traffic loads, Network Coding can be used to improve a network's | |||
throughput, efficiency, latency, and scalability, as well as | throughput, efficiency, latency, and scalability, as well as | |||
resilience to partition, attacks, and eavesdropping, as compared to | resilience to partition, attacks, and eavesdropping, as compared to | |||
traditional methods. DetNet could utilized Network coding as an | traditional methods. DetNet could use Network coding as an | |||
alternative to other protection means. Network coding is often | alternative to other protection means. Network coding is often | |||
applied in wireless networks and is being explored for other network | applied in wireless networks and is being explored for other network | |||
types. | types. | |||
3.6.1.5. Load sharing | 3.5.1.5. Load sharing | |||
Use of packet-by-packet distribution of the same DetNet flow over | Use of packet-by-packet distribution of the same DetNet flow over | |||
multiple paths is not recommended except for the cases listed above | multiple paths is not recommended except for the cases listed above | |||
where PREOF is utilized to improve protection of traffic and maintain | where PREOF is utilized to improve protection of traffic and maintain | |||
order. Packet by packet load sharing, e.g., via ECMP or UCMP, | order. Packet by packet load sharing, e.g., via ECMP or UCMP, | |||
impacts ordering and possibly jitter. | impacts ordering and possibly jitter. | |||
3.6.1.6. Troubleshooting | 3.5.1.6. Troubleshooting | |||
Detnet leverages many different forwarding sub-layers, each of which | Detnet leverages many different forwarding sub-layers, each of which | |||
supports various tools to troubleshoot connectivity, for example | supports various tools to troubleshoot connectivity, for example | |||
identification of misbehaving flows. The DetNet Service layer can | identification of misbehaving flows. The DetNet Service layer can | |||
leverage existing mechanisms to troubleshoot or monitor flows, such | leverage existing mechanisms to troubleshoot or monitor flows, such | |||
as those in use by IP and MPLS networks. At the Application layer a | as those in use by IP and MPLS networks. At the Application layer a | |||
client of a DetNet service can use existing techniques to detect and | client of a DetNet service can use existing techniques to detect and | |||
monitor delay and loss. | monitor delay and loss. | |||
3.6.1.7. Flow recognition for analytics | 3.5.1.7. Flow recognition for analytics | |||
Network analytics can be inherited from the technologies of the | Network analytics can be inherited from the technologies of the | |||
Service and Forwarding sub-layers. At the DetNet service edge, | Service and Forwarding sub-layers. At the DetNet service edge, | |||
packet and bit counters (e.g. sent, received, dropped, and out-of- | packet and bit counters (e.g. sent, received, dropped, and out-of- | |||
sequence) can be maintained. | sequence) can be maintained. | |||
3.6.1.8. Correlation of events with flows | 3.5.1.8. Correlation of events with flows | |||
The provider of a DetNet service may provide other capabilities to | The provider of a DetNet service may provide other capabilities to | |||
monitor flows, such as more detailed loss statistics and time | monitor flows, such as more detailed loss statistics and time | |||
stamping of events. The details of these capabilities are currently | stamping of events. The details of these capabilities are out of | |||
out of scope for this document. | scope for this document. | |||
3.6.2. Service Protection | 3.5.2. Service Protection | |||
Service protection allow DetNet services to increase reliability and | Service protection allows DetNet services to increase reliability and | |||
maintain a DetNet Service Assurance in the case of network congestion | maintain a DetNet Service Assurance in the case of network congestion | |||
or network failure. Detnet relies on the underlying technology | or network failure. Detnet relies on the underlying technology | |||
capabilities for various protection schemes. Protection schemes | capabilities for various protection schemes. Protection schemes | |||
enable partial or complete coverage of the network paths and active | enable partial or complete coverage of the network paths and active | |||
protection with combinations of PRF, PEF, and POF. | protection with combinations of PRF, PEF, and POF. | |||
3.6.2.1. Linear Service Protection | 3.5.2.1. Linear Service Protection | |||
An example DetNet MPLS network fragment and packet flow is | An example DetNet MPLS network fragment and packet flow is | |||
illustrated in Figure 3. | illustrated in Figure 3. | |||
1 1.1 1.1 1.2.1 1.2.1 1.2.2 | 1 1.1 1.1 1.2.1 1.2.1 1.2.2 | |||
CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 | CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 | |||
\ 1.2.1 / / | \ 1.2.1 / / | |||
\1.2 /-----+ / | \1.2 /-----+ / | |||
+------R4------------------------+ | +------R4------------------------+ | |||
1.2.2 | 1.2.2 | |||
Figure 3: Example Packet Flow in DetNet protected Network | Figure 3: Example Packet Flow in DetNet Protected Network | |||
In Figure 3 the numbers are used to identify the instance of a | In Figure 3 the numbers are used to identify the instance of a | |||
packet. Packet 1 is the original packet, and packets 1.1, and 1.2 | packet. Packet 1 is the original packet, and packets 1.1, and 1.2 | |||
are two first generation copies of packet 1. Packet 1.2.1 is a | are two first generation copies of packet 1. Packet 1.2.1 is a | |||
second generation copy of packet 1.2 etc. Note that these numbers | second generation copy of packet 1.2, etc. Note that these numbers | |||
never appear in the packet, and are not to be confused with sequence | never appear in the packet, and are not to be confused with sequence | |||
numbers, labels or any other identifier that appears in the packet. | numbers, labels or any other identifier that appears in the packet. | |||
They simply indicate the generation number of the original packet so | They simply indicate the generation number of the original packet so | |||
that its passage through the network fragment can be identified to | that its passage through the network fragment can be identified to | |||
the reader. | the reader. | |||
Customer Equipment CE1 sends a packet into the DetNet enabled | Customer Equipment CE1 sends a packet into the DetNet enabled | |||
network. This is packet (1). Edge Node EN1 encapsulates the packet | network. This is packet (1). Edge Node EN1 encapsulates the packet | |||
as a DetNet Packet and sends it to Relay node R1 (packet 1.1). EN1 | as a DetNet packet and sends it to Relay node R1 (packet 1.1). EN1 | |||
makes a copy of the packet (1.2), encapsulates it and sends this copy | makes a copy of the packet (1.2), encapsulates it and sends this copy | |||
to Relay node R4. | to Relay node R4. | |||
Note that along the path from EN1 to R1 there may be zero or more | Note that along the path from EN1 to R1 there may be zero or more | |||
nodes which, for clarity, are not shown. The same is true for any | nodes which, for clarity, are not shown. The same is true for any | |||
other path between two DetNet entities shown in Figure 3 . | other path between two DetNet entities shown in Figure 3. | |||
Relay node R4 has been configured to send one copy of the packet to | Relay node R4 has been configured to send one copy of the packet to | |||
Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet | Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet | |||
1.2.2). | 1.2.2). | |||
R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, | R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, | |||
having been configured to perform packet elimination on this DetNet | having been configured to perform packet elimination on this DetNet | |||
flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of | flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of | |||
no further use and so is discarded by R2. | no further use and so is discarded by R2. | |||
Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives | Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives | |||
packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips | packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips | |||
any DetNet encapsulation from packet copy 1.2.2 and forwards the | any DetNet encapsulation from packet copy 1.2.2 and forwards the | |||
packet to CE2. When EN2 receives the later packet copy 1.2.1 this is | packet to CE2. When EN2 receives the later packet copy 1.2.1 this is | |||
discarded. | discarded. | |||
The above is of course illustrative of many network scenarios that | The above is of course illustrative of many network scenarios that | |||
can be configured. | can be configured. | |||
This example also illustrates 1:1 protection scheme meaning there is | This example also illustrates 1:1 protection scheme meaning there is | |||
traffic over each segment of the end to end path. Local DetNet relay | traffic over each segment of the end-to-end path. Local DetNet relay | |||
nodes determine which packets are eliminated and which packets are | nodes determine which packets are eliminated and which packets are | |||
forwarded. A 1+1 scheme where only one path is used for traffic at a | forwarded. A 1+1 scheme where only one path is used for traffic at a | |||
time, could use the same topology. In this case there is no PRF | time could use the same topology. In this case there is no PRF | |||
function and traffic is switched upon detection of failure. An OAM | function and traffic is switched upon detection of failure. An OAM | |||
scheme that monitors the paths detects the loss of path or traffic is | scheme that monitors the paths to detect the loss of path or traffic | |||
required to initiate the switch. A POF may still be used in this | is required to initiate the switch. A POF may still be used in this | |||
case to prevent misordering of packets. In both cases the protection | case to prevent misordering of packets. In both cases the protection | |||
paths are established and maintained for the duration of the DetNet | paths are established and maintained for the duration of the DetNet | |||
service. | service. | |||
3.6.2.2. Path Differential Delay | 3.5.2.2. Path Differential Delay | |||
In the preceding example, proper working of duplicate elimination and | In the preceding example, proper working of duplicate elimination and | |||
reordering of packets are dependent on the number of out-of-order | reordering of packets are dependent on the number of out-of-order | |||
packets that can be buffered and the delay difference of arriving | packets that can be buffered and the delay difference of arriving | |||
packets. DetNet uses flow specific requirements (e.g., maximum | packets. DetNet uses flow-specific requirements (e.g., maximum | |||
number of out-of-order packets, maximum latency of the flow, etc.) | number of out-of-order packets, maximum latency of the flow) for | |||
for configuration of POF related buffers. If the differential delay | configuration of POF-related buffers. If the differential delay | |||
between paths is excessively large or there is excessive mis-ordering | between paths is excessively large or there is excessive mis-ordering | |||
of the packets, then packets may be dropped instead of being | of the packets, then packets may be dropped instead of being | |||
reordered. Likewise, PEF uses the sequence number to identify | reordered. Likewise, PEF uses the sequence number to identify | |||
duplicate packets, and large differential delays combined with high | duplicate packets, and large differential delays combined with high | |||
numbers of packets may exceed the ability of the PEF to work | numbers of packets may exceed the ability of the PEF to work | |||
properly. | properly. | |||
3.6.2.3. Ring Service Protection | 3.5.2.3. Ring Service Protection | |||
Ring protection may also be supported if the underlying technology | Ring protection may also be supported if the underlying technology | |||
supports it. Many of the same concepts apply however rings are | supports it. Many of the same concepts apply, however rings are | |||
normally 1+1 protection for data efficiency reasons. [RFC8227] is an | normally 1+1 protection for data efficiency reasons. [RFC8227] is an | |||
example of MPLS-TP data plane that supports Ring protection. | example of MPLS-TP data plane that supports Ring protection. | |||
3.6.3. Aggregation Considerations | 3.5.3. Aggregation Considerations | |||
The DetNet data plane also allows for the aggregation of DetNet | The DetNet data plane also allows for the aggregation of DetNet | |||
flows, which can improve scalability by reducing the per-hop state. | flows, which can improve scalability by reducing the per-hop state. | |||
How this is accomplished is data plane or control plane dependent. | How this is accomplished is data plane or control plane dependent. | |||
When DetNet flows are aggregated, transit nodes provide service to | When DetNet flows are aggregated, transit nodes provide service to | |||
the aggregate and not on a per-DetNet flow basis. When aggregating | the aggregate and not on a per-DetNet flow basis. When aggregating | |||
DetNet flows the flows should be compatible i.e. the same or very | DetNet flows, the flows should be compatible, i.e., the same or very | |||
similar QoS and CoS characteristics. In this case, nodes performing | similar QoS and CoS characteristics. In this case, nodes performing | |||
aggregation will ensure that per-flow service requirements are | aggregation will ensure that per-flow service requirements are | |||
achieved. | achieved. | |||
If bandwidth reservations are used, the sum of the reservations | If bandwidth reservations are used, the reservation should be the sum | |||
should be the sum of all the individual reservations; in other words, | of all the individual reservations; in other words, the reservations | |||
the reservations should not add up to an over-subscription of | should not add up to an over-subscription of bandwidth reservation. | |||
bandwidth reservation. If maximum delay bounds are used, the system | If maximum delay bounds are used, the system should ensure that the | |||
should ensure that the aggregate does not exceed the delay bounds of | aggregate does not exceed the delay bounds of the individual flows. | |||
the individual flows. | ||||
When an encapsulation is used the choice of reserving a maximum | When an encapsulation is used, the choice of reserving a maximum | |||
resource level and then tracking the services in the aggregated | resource level and then tracking the services in the aggregated | |||
service or adjusting the aggregated resources as the services are | service or adjusting the aggregated resources as the services are | |||
added is implementation and technology specific. | added is implementation and technology specific. | |||
DetNet flows at edges must be able to handle rejection to an | DetNet flows at edges must be able to handle rejection to an | |||
aggregation group due to lack of resources as well as conditions | aggregation group due to lack of resources as well as conditions | |||
where requirements are not satisfied. | where requirements are not satisfied. | |||
3.6.3.1. IP Aggregation | 3.5.3.1. IP Aggregation | |||
IP aggregation has both data plane and controller plane aspects. For | IP aggregation has both data plane and controller plane aspects. For | |||
the data plane, flows may be aggregated for treatment based on shared | the data plane, flows may be aggregated for treatment based on shared | |||
characteristics such as 6-tuple. Alternatively, an IP encapsulation | characteristics such as 6-tuple. Alternatively, an IP encapsulation | |||
may be used to tunnel an aggregate number of DetNet Flows between | may be used to tunnel an aggregate number of DetNet Flows between | |||
relay nodes. | relay nodes. | |||
3.6.3.2. MPLS Aggregation | 3.5.3.2. MPLS Aggregation | |||
MPLS aggregation also has data plane and controller plane aspects. | MPLS aggregation also has data plane and controller plane aspects. | |||
MPLS flows are often tunneled in a forwarding sub-layer, under the | MPLS flows are often tunneled in a forwarding sub-layer, under the | |||
reservation associated with that MPLS tunnel. | reservation associated with that MPLS tunnel. | |||
3.6.4. End-System-Specific Considerations | 3.5.4. End-System-Specific Considerations | |||
Data-flows requiring DetNet service are generated and terminated on | Data-flows requiring DetNet service are generated and terminated on | |||
end-systems. Encapsulation depends on the application and its | end-systems. Encapsulation depends on the application and its | |||
preferences. For example, in a DetNet MPLS domain the sub-layer | preferences. For example, in a DetNet MPLS domain the sub-layer | |||
functions use the d-CWs, S-Labels and F-Labels to provide DetNet | functions use the d-CWs, S-Labels and F-Labels to provide DetNet | |||
services. However, an application may exchange further flow related | services. However, an application may exchange further flow related | |||
parameters (e.g., time-stamp), which are not provided by DetNet | parameters (e.g., time-stamp), which are not provided by DetNet | |||
functions. | functions. | |||
As a general rule, DetNet domains are capable of forwarding any | As a general rule, DetNet domains are capable of forwarding any | |||
skipping to change at page 15, line 26 ¶ | skipping to change at page 15, line 5 ¶ | |||
/ \ __/ \ | / \ __/ \ | |||
+-----+ \__ DetNet MPLS domain / \ | +-----+ \__ DetNet MPLS domain / \ | |||
| X | \ __ / +-----+ | | X | \ __ / +-----+ | |||
+-----+ \_______/ \_____/ | X | | +-----+ \_______/ \_____/ | X | | |||
| IP | +-----+ | | IP | +-----+ | |||
+-----+ | IP | | +-----+ | IP | | |||
+-----+ | +-----+ | |||
Figure 4: End-Systems and The DetNet MPLS Domain | Figure 4: End-Systems and The DetNet MPLS Domain | |||
3.6.5. Sub-Network Considerations | 3.5.5. Sub-Network Considerations | |||
Any of the DetNet service types may be transported by another DetNet | Any of the DetNet service types may be transported by another DetNet | |||
service. MPLS nodes may interconnected by different sub-network | service. MPLS nodes may be interconnected by different sub-network | |||
technologies, which may include point-to-point links. Each of these | technologies, which may include point-to-point links. Each of these | |||
sub-network technologies need to provide appropriate service to | sub-network technologies needs to provide appropriate service to | |||
DetNet flows. In some cases, e.g., on dedicated point-to-point links | DetNet flows. In some cases, e.g., on dedicated point-to-point links | |||
or TDM technologies, all that is required is for a DetNet node to | or TDM technologies, all that is required is for a DetNet node to | |||
appropriately queue its output traffic. In other cases, DetNet nodes | appropriately queue its output traffic. In other cases, DetNet nodes | |||
will need to map DetNet flows to the flow semantics (i.e., | will need to map DetNet flows to the flow semantics (i.e., | |||
identifiers) and mechanisms used by an underlying sub-network | identifiers) and mechanisms used by an underlying sub-network | |||
technology. Figure 5 shows several examples of header formats that | technology. Figure 5 shows several examples of header formats that | |||
can be used to carry DetNet MPLS flows over different sub-network | can be used to carry DetNet MPLS flows over different sub-network | |||
technologies. L2 represent a generic layer-2 encapsulation that | technologies. L2 represents a generic layer-2 encapsulation that | |||
might be used on a point-to-point link. TSN represents the | might be used on a point-to-point link. TSN represents the | |||
encapsulation used on an IEEE 802.1 TSN network, as described in | encapsulation used on an IEEE 802.1 TSN network, as described in | |||
[I-D.ietf-detnet-mpls-over-tsn]. UDP/IP represents the encapsulation | [I-D.ietf-detnet-mpls-over-tsn]. UDP/IP represents the encapsulation | |||
used on a DetNet IP PSN, as described in | used on a DetNet IP PSN, as described in | |||
[I-D.ietf-detnet-mpls-over-udp-ip]. | [I-D.ietf-detnet-mpls-over-udp-ip]. | |||
+------+ +------+ +------+ | +------+ +------+ +------+ | |||
App-Flow | X | | X | | X | | App-Flow | X | | X | | X | | |||
+-----+======+--+======+--+======+-----+ | +-----+======+--+======+--+======+-----+ | |||
DetNet-MPLS | d-CW | | d-CW | | d-CW | | DetNet-MPLS | d-CW | | d-CW | | d-CW | | |||
skipping to change at page 16, line 30 ¶ | skipping to change at page 15, line 50 ¶ | |||
4. Controller Plane (Management and Control) Considerations | 4. Controller Plane (Management and Control) Considerations | |||
4.1. DetNet Controller Plane Requirements | 4.1. DetNet Controller Plane Requirements | |||
The Controller Plane corresponds to the aggregation of the Control | The Controller Plane corresponds to the aggregation of the Control | |||
and Management Planes discussed in [RFC7426] and [RFC8655]. While | and Management Planes discussed in [RFC7426] and [RFC8655]. While | |||
more details of any DetNet controller plane are out of the scope of | more details of any DetNet controller plane are out of the scope of | |||
this document, there are particular considerations and requirements | this document, there are particular considerations and requirements | |||
for such that result from the unique characteristics of the DetNet | for such that result from the unique characteristics of the DetNet | |||
architecture [RFC8655] and data plane as defined herein. | architecture and data plane as defined herein. | |||
The primary requirements of the DetNet controller plane are that it | The primary requirements of the DetNet controller plane are that it | |||
must be able to: | must be able to: | |||
o Instantiate DetNet flows in a DetNet domain (which may include | o Instantiate DetNet flows in a DetNet domain (which may include | |||
some or all of explicit path determination, link bandwidth | some or all of explicit path determination, link bandwidth | |||
reservations, restricting flows to IEEE 802.1 TSN links, node | reservations, restricting flows to IEEE 802.1 TSN links, node | |||
buffer and other resource reservations, specification of required | buffer and other resource reservations, specification of required | |||
queuing disciplines along the path, ability to manage | queuing disciplines along the path, ability to manage | |||
bidirectional flows, etc.) as needed for a flow. | bidirectional flows, etc.) as needed for a flow. | |||
skipping to change at page 17, line 41 ¶ | skipping to change at page 17, line 13 ¶ | |||
if link, node, or management equipment failures occur. While a | if link, node, or management equipment failures occur. While a | |||
detailed analysis of the control plane alternatives is out of the | detailed analysis of the control plane alternatives is out of the | |||
scope of this document, the requirements from this document can be | scope of this document, the requirements from this document can be | |||
used as the basis of a later analysis of the alternatives. | used as the basis of a later analysis of the alternatives. | |||
4.2. Generic Controller Plane Considerations | 4.2. Generic Controller Plane Considerations | |||
This section covers control plane considerations that are independent | This section covers control plane considerations that are independent | |||
of the data plane technology used for DetNet service delivery. | of the data plane technology used for DetNet service delivery. | |||
While management plane and control planes are traditionally | While the management plane and control planes are traditionally | |||
considered separately, from the Data Plane perspective there is no | considered separately, from the data plane perspective there is no | |||
practical difference based on the origin of flow provisioning | practical difference based on the origin of flow provisioning | |||
information, and the DetNet architecture [RFC8655] refers to these | information, and the DetNet architecture [RFC8655] refers to these | |||
collectively as the 'Controller Plane'. This document therefore does | collectively as the 'Controller Plane'. This document therefore does | |||
not distinguish between information provided by distributed control | not distinguish between information provided by distributed control | |||
plane protocols, e.g., RSVP-TE [RFC3209] and [RFC3473], or by | plane protocols, e.g., RSVP-TE [RFC3209] and [RFC3473], or by | |||
centralized network management mechanisms, e.g., RestConf [RFC8040], | centralized network management mechanisms, e.g., RestConf [RFC8040], | |||
YANG [RFC7950], and the Path Computation Element Communication | YANG [RFC7950], and the Path Computation Element Communication | |||
Protocol (PCEP) [I-D.ietf-pce-pcep-extension-for-pce-controller] or | Protocol (PCEP) [I-D.ietf-pce-pcep-extension-for-pce-controller] or | |||
any combination thereof. Specific considerations and requirements | any combination thereof. Specific considerations and requirements | |||
for the DetNet Controller Plane are discussed in Section 4.1. | for the DetNet Controller Plane are discussed in Section 4.1. | |||
Each respective data plane document also covers the control plane | Each respective data plane document also covers the control plane | |||
considerations for that technology. For example [I-D.ietf-detnet-ip] | considerations for that technology. For example, | |||
covers IP control plane normative considerations and | [I-D.ietf-detnet-ip] covers IP control plane normative considerations | |||
[I-D.ietf-detnet-mpls] covers MPLS control plane normative | and [I-D.ietf-detnet-mpls] covers MPLS control plane normative | |||
considerations. | considerations. | |||
4.2.1. Flow Aggregation Control | 4.2.1. Flow Aggregation Control | |||
Flow aggregation means that multiple App-flows are served by a single | Flow aggregation means that multiple app-flows are served by a single | |||
new DetNet flow. There are many techniques to achieve aggregation, | new DetNet flow. There are many techniques to achieve aggregation. | |||
for example in case of IP, it can be grouping of IP flows that share | For example, in the case of IP, IP flows that share 6-tuple | |||
6-tuple attributes or flow identifiers at the DetNet sub-layer. | attributes or flow identifiers at the DetNet sub-layer can be | |||
Another example includes aggregation accomplished through the use of | grouped. Another example includes aggregation accomplished through | |||
hierarchical LSPs in MPLS and tunnels. | the use of hierarchical LSPs in MPLS and tunnels. | |||
Control of aggregation involves a set of procedures listed here. | Control of aggregation involves a set of procedures listed here. | |||
Aggregation may use some or all of these capabilities and the order | Aggregation may use some or all of these capabilities and the order | |||
may vary: | may vary: | |||
o Traffic engineering resource collection and distribution: | o Traffic engineering resource collection and distribution: | |||
Available resources are tracked through control plane or | Available resources are tracked through control plane or | |||
management plane databases and distributed amongst controllers | management plane databases and distributed amongst controllers | |||
or nodes that can manage resources. | or nodes that can manage resources. | |||
o Path computation and resource allocation: | o Path computation and resource allocation: | |||
When DetNet services are provisioned or requested one or more | When DetNet services are provisioned or requested, one or more | |||
paths meeting the requirements are selected and the resources | paths meeting the requirements are selected and the resources | |||
verified and recorded. | verified and recorded. | |||
o Resource assignment and data plane co-ordination: | o Resource assignment and data plane co-ordination: | |||
The assignment of resources along the path depends on the | The assignment of resources along the path depends on the | |||
technology and it includes assignment of specific links and | technology and includes assignment of specific links, | |||
coordination of the queuing and other traffic management | coordination of queueing, and other traffic management | |||
capabilities such as policing and shaping. | capabilities such as policing and shaping. | |||
o Assigned Resource recording and updating: | o Assigned Resource recording and updating: | |||
Depending on the specific technology, the assigned resources | Depending on the specific technology, the assigned resources | |||
are updated and distributed in the databases, preventing over- | are updated and distributed in the databases, preventing over- | |||
subscription. | subscription. | |||
4.2.2. Explicit Routes | 4.2.2. Explicit Routes | |||
skipping to change at page 21, line 10 ¶ | skipping to change at page 20, line 28 ¶ | |||
points for these functions in one direction may not match the optimal | points for these functions in one direction may not match the optimal | |||
points in the other, due to network and traffic constraints. | points in the other, due to network and traffic constraints. | |||
Furthermore, due to the per packet service protection nature, | Furthermore, due to the per packet service protection nature, | |||
bidirectional forwarding per packet may not be ensured. The first | bidirectional forwarding per packet may not be ensured. The first | |||
packet of received member flows is selected by the elimination | packet of received member flows is selected by the elimination | |||
function independently of which path it has taken through the | function independently of which path it has taken through the | |||
network. | network. | |||
Control and management mechanisms need to support bidirectional | Control and management mechanisms need to support bidirectional | |||
flows, but the specification of such mechanisms are out of scope of | flows, but the specification of such mechanisms are out of scope of | |||
this document. An example control plane solution for MPLS can be | this document. Example control plane solutions for MPLS can be found | |||
found in [RFC3473] , [RFC6387] and [RFC7551]. These requirements are | in [RFC3473] , [RFC6387] and [RFC7551]. These requirements are | |||
included in Section 4.1. | included in Section 4.1. | |||
4.3. Packet Replication, Elimination, and Ordering (PREOF) | 4.3. Packet Replication, Elimination, and Ordering (PREOF) | |||
The controller plane protocol solution required for managing the | The controller plane protocol solution required for managing the | |||
PREOF processing is outside the scope of this document. That said, | PREOF processing is outside the scope of this document. That said, | |||
it should be noted that the ability to determine, for a particular | it should be noted that the ability to determine, for a particular | |||
flow, optimal packet replication and elimination points in the DetNet | flow, optimal packet replication and elimination points in the DetNet | |||
domain requires explicit support. There may be capabilities that can | domain requires explicit support. There may be capabilities that can | |||
be used, or extended, for example GMPLS end-to-end recovery [RFC4872] | be used, or extended, for example GMPLS end-to-end recovery [RFC4872] | |||
and GMPLS segment recovery [RFC4873]. | and GMPLS segment recovery [RFC4873]. | |||
5. Security Considerations | 5. Security Considerations | |||
Security considerations for DetNet are described in detail in | Security considerations for DetNet are described in detail in | |||
[I-D.ietf-detnet-security]. General security considerations are | [I-D.ietf-detnet-security]. General security considerations for | |||
described in [RFC8655]. This section considers general security | DetNet architecture are described in [RFC8655]. This section | |||
considerations applicable to all data planes. | considers general security considerations applicable to all data | |||
planes. | ||||
Security aspects which are unique to DetNet are those whose aim is to | Security aspects which are unique to DetNet are those whose aim is to | |||
provide the specific quality of service aspects of DetNet, which are | provide the specific quality of service aspects of DetNet, which are | |||
primarily to deliver data flows with extremely low packet loss rates | primarily to deliver data flows with extremely low packet loss rates | |||
and bounded end-to-end delivery latency. | and bounded end-to-end delivery latency. | |||
The primary considerations for the data plane is to maintain | The primary consideration for the data plane is to maintain integrity | |||
integrity of data and delivery of the associated DetNet service | of data and delivery of the associated DetNet service traversing the | |||
traversing the DetNet network. Application flows can be protected | DetNet network. Application flows can be protected through whatever | |||
through whatever means is provided by the underlying technology. For | means is provided by the underlying technology. For example, | |||
example, encryption may be used, such as that provided by IPSec | encryption may be used, such as that provided by IPSec [RFC4301] for | |||
[RFC4301] for IP flows and/or by an underlying sub-net using MACSec | IP flows and/or by an underlying sub-net using MACSec | |||
[IEEE802.1AE-2018] for Ethernet (Layer-2) flows. | [IEEE802.1AE-2018] for Ethernet (Layer-2) flows. | |||
At the management and control level DetNet flows are identified on a | At the management and control level DetNet flows are identified on a | |||
per-flow basis, which may provide controller plane attackers with | per-flow basis, which may provide controller plane attackers with | |||
additional information about the data flows (when compared to | additional information about the data flows (when compared to | |||
controller planes that do not include per-flow identification). This | controller planes that do not include per-flow identification). This | |||
is an inherent property of DetNet which has security implications | is an inherent property of DetNet which has security implications | |||
that should be considered when determining if DetNet is a suitable | that should be considered when determining if DetNet is a suitable | |||
technology for any given use case. | technology for any given use case. | |||
To provide uninterrupted availability of the DetNet service, | To provide uninterrupted availability of the DetNet service, | |||
provisions can be made against DOS attacks and delay attacks. To | provisions can be made against DOS attacks and delay attacks. To | |||
protect against DOS attacks, excess traffic due to malicious or | protect against DOS attacks, excess traffic due to malicious or | |||
malfunctioning devices can be prevented or mitigated, for example | malfunctioning devices can be prevented or mitigated, for example | |||
through the use of existing mechanism such as policing and shaping | through the use of existing mechanisms such as policing and shaping | |||
applied at the input of a DetNet domain. To prevent DetNet packets | applied at the input of a DetNet domain. To prevent DetNet packets | |||
from being delayed by an entity external to a DetNet domain, DetNet | from being delayed by an entity external to a DetNet domain, DetNet | |||
technology definition can allow for the mitigation of Man-In-The- | technology definition can allow for the mitigation of Man-In-The- | |||
Middle attacks, for example through use of authentication and | Middle attacks, for example through use of authentication and | |||
authorization of devices within the DetNet domain. | authorization of devices within the DetNet domain. | |||
In order to prevent or mitigate DetNet attacks on other networks via | In order to prevent or mitigate DetNet attacks on other networks via | |||
flow escape, edge devices can for example use existing mechanism such | flow escape, edge devices can for example use existing mechanism such | |||
as policing and shaping applied at the output of a DetNet domain. | as policing and shaping applied at the output of a DetNet domain. | |||
skipping to change at page 23, line 20 ¶ | skipping to change at page 22, line 43 ¶ | |||
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, | [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, | |||
"Deterministic Networking Architecture", RFC 8655, | "Deterministic Networking Architecture", RFC 8655, | |||
DOI 10.17487/RFC8655, October 2019, | DOI 10.17487/RFC8655, October 2019, | |||
<https://www.rfc-editor.org/info/rfc8655>. | <https://www.rfc-editor.org/info/rfc8655>. | |||
9.2. Informative References | 9.2. Informative References | |||
[I-D.ietf-detnet-flow-information-model] | [I-D.ietf-detnet-flow-information-model] | |||
Farkas, J., Varga, B., Cummings, R., Jiang, Y., and D. | Farkas, J., Varga, B., Cummings, R., Jiang, Y., and D. | |||
Fedyk, "DetNet Flow Information Model", draft-ietf-detnet- | Fedyk, "DetNet Flow Information Model", draft-ietf-detnet- | |||
flow-information-model-06 (work in progress), October | flow-information-model-07 (work in progress), March 2020. | |||
2019. | ||||
[I-D.ietf-detnet-ip] | [I-D.ietf-detnet-ip] | |||
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., | Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., | |||
Bryant, S., and J. Korhonen, "DetNet Data Plane: IP", | and S. Bryant, "DetNet Data Plane: IP", draft-ietf-detnet- | |||
draft-ietf-detnet-ip-04 (work in progress), November 2019. | ip-05 (work in progress), February 2020. | |||
[I-D.ietf-detnet-mpls] | [I-D.ietf-detnet-mpls] | |||
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., | Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., | |||
Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", | Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", | |||
draft-ietf-detnet-mpls-04 (work in progress), November | draft-ietf-detnet-mpls-05 (work in progress), February | |||
2019. | 2020. | |||
[I-D.ietf-detnet-mpls-over-tsn] | [I-D.ietf-detnet-mpls-over-tsn] | |||
Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet | Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet | |||
Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking | Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking | |||
(TSN)", draft-ietf-detnet-mpls-over-tsn-01 (work in | (TSN)", draft-ietf-detnet-mpls-over-tsn-02 (work in | |||
progress), October 2019. | progress), March 2020. | |||
[I-D.ietf-detnet-mpls-over-udp-ip] | [I-D.ietf-detnet-mpls-over-udp-ip] | |||
Varga, B., Farkas, J., Berger, L., Malis, A., Bryant, S., | Varga, B., Farkas, J., Berger, L., Malis, A., and S. | |||
and J. Korhonen, "DetNet Data Plane: MPLS over UDP/IP", | Bryant, "DetNet Data Plane: MPLS over UDP/IP", draft-ietf- | |||
draft-ietf-detnet-mpls-over-udp-ip-04 (work in progress), | detnet-mpls-over-udp-ip-05 (work in progress), February | |||
November 2019. | 2020. | |||
[I-D.ietf-detnet-security] | [I-D.ietf-detnet-security] | |||
Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell, | Mizrahi, T. and E. Grossman, "Deterministic Networking | |||
J., Austad, H., and N. Finn, "Deterministic Networking | ||||
(DetNet) Security Considerations", draft-ietf-detnet- | (DetNet) Security Considerations", draft-ietf-detnet- | |||
security-07 (work in progress), January 2020. | security-09 (work in progress), March 2020. | |||
[I-D.ietf-pce-pcep-extension-for-pce-controller] | [I-D.ietf-pce-pcep-extension-for-pce-controller] | |||
Zhao, Q., Li, Z., Negi, M., and C. Zhou, "PCEP Procedures | Zhao, Q., Li, Z., Negi, M., Peng, S., and C. Zhou, "PCEP | |||
and Protocol Extensions for Using PCE as a Central | Procedures and Protocol Extensions for Using PCE as a | |||
Controller (PCECC) of LSPs", draft-ietf-pce-pcep- | Central Controller (PCECC) of LSPs", draft-ietf-pce-pcep- | |||
extension-for-pce-controller-03 (work in progress), | extension-for-pce-controller-04 (work in progress), March | |||
November 2019. | 2020. | |||
[I-D.ietf-spring-srv6-network-programming] | [I-D.ietf-spring-srv6-network-programming] | |||
Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., | Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., | |||
Matsushima, S., and Z. Li, "SRv6 Network Programming", | Matsushima, S., and Z. Li, "SRv6 Network Programming", | |||
draft-ietf-spring-srv6-network-programming-08 (work in | draft-ietf-spring-srv6-network-programming-15 (work in | |||
progress), January 2020. | progress), March 2020. | |||
[IEEE802.1AE-2018] | [IEEE802.1AE-2018] | |||
IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC | IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC | |||
Security (MACsec)", 2018, | Security (MACsec)", 2018, | |||
<https://ieeexplore.ieee.org/document/8585421>. | <https://ieeexplore.ieee.org/document/8585421>. | |||
[IEEE802.1TSNTG] | [IEEE802.1TSNTG] | |||
IEEE Standards Association, "IEEE 802.1 Time-Sensitive | IEEE Standards Association, "IEEE 802.1 Time-Sensitive | |||
Networking Task Group", <http://www.ieee802.org/1/tsn>. | Networking Task Group", <http://www.ieee802.org/1/tsn>. | |||
skipping to change at page 26, line 27 ¶ | skipping to change at page 26, line 4 ¶ | |||
Email: balazs.a.varga@ericsson.com | Email: balazs.a.varga@ericsson.com | |||
Janos Farkas | Janos Farkas | |||
Ericsson | Ericsson | |||
Magyar Tudosok krt. 11. | Magyar Tudosok krt. 11. | |||
Budapest 1117 | Budapest 1117 | |||
Hungary | Hungary | |||
Email: janos.farkas@ericsson.com | Email: janos.farkas@ericsson.com | |||
Lou Berger | Lou Berger | |||
LabN Consulting, L.L.C. | LabN Consulting, L.L.C. | |||
Email: lberger@labn.net | Email: lberger@labn.net | |||
Andrew G. Malis | Andrew G. Malis | |||
Independent | Malis Consulting | |||
Email: agmalis@gmail.com | Email: agmalis@gmail.com | |||
Stewart Bryant | Stewart Bryant | |||
Futurewei Technologies | Futurewei Technologies | |||
Email: stewart.bryant@gmail.com | Email: stewart.bryant@gmail.com | |||
End of changes. 96 change blocks. | ||||
204 lines changed or deleted | 198 lines changed or added | |||
This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |