draft-ietf-detnet-data-plane-framework-06.txt   rfc8938.txt 
DetNet B. Varga, Ed. Internet Engineering Task Force (IETF) B. Varga, Ed.
Internet-Draft J. Farkas Request for Comments: 8938 J. Farkas
Intended status: Informational Ericsson Category: Informational Ericsson
Expires: November 7, 2020 L. Berger ISSN: 2070-1721 L. Berger
LabN Consulting, L.L.C. LabN Consulting, L.L.C.
A. Malis A. Malis
Malis Consulting Malis Consulting
S. Bryant S. Bryant
Futurewei Technologies Futurewei Technologies
May 6, 2020 November 2020
DetNet Data Plane Framework Deterministic Networking (DetNet) Data Plane Framework
draft-ietf-detnet-data-plane-framework-06
Abstract Abstract
This document provides an overall framework for the DetNet data This document provides an overall framework for the Deterministic
plane. It covers concepts and considerations that are generally Networking (DetNet) data plane. It covers concepts and
common to any Deterministic Networking data plane specification. It considerations that are generally common to any DetNet data plane
describes related controller plane considerations as well. specification. It describes related Controller Plane considerations
as well.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for informational purposes.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.
This Internet-Draft will expire on November 7, 2020. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8938.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology
2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 2.1. Terms Used in This Document
2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 2.2. Abbreviations
3. DetNet Data Plane Overview . . . . . . . . . . . . . . . . . 4 3. Overview of the DetNet Data Plane
3.1. Data Plane Characteristics . . . . . . . . . . . . . . . 6 3.1. Data Plane Characteristics
3.1.1. Data Plane Technology . . . . . . . . . . . . . . . . 6 3.1.1. Data Plane Technology
3.1.2. Encapsulation . . . . . . . . . . . . . . . . . . . . 6 3.1.2. Encapsulation
3.2. DetNet-specific Metadata . . . . . . . . . . . . . . . . 7 3.2. DetNet-Specific Metadata
3.3. DetNet IP Data Plane . . . . . . . . . . . . . . . . . . 8 3.3. DetNet IP Data Plane
3.4. DetNet MPLS Data Plane . . . . . . . . . . . . . . . . . 8 3.4. DetNet MPLS Data Plane
3.5. Further DetNet Data Plane Considerations . . . . . . . . 9 3.5. Further DetNet Data Plane Considerations
3.5.1. Per Flow Related Functions . . . . . . . . . . . . . 9 3.5.1. Functions Provided on a Per-Flow Basis
3.5.2. Service Protection . . . . . . . . . . . . . . . . . 11 3.5.2. Service Protection
3.5.3. Aggregation Considerations . . . . . . . . . . . . . 13 3.5.3. Aggregation Considerations
3.5.4. End-System-Specific Considerations . . . . . . . . . 14 3.5.4. End-System-Specific Considerations
3.5.5. Sub-Network Considerations . . . . . . . . . . . . . 15 3.5.5. Sub-network Considerations
4. Controller Plane (Management and Control) 4. Controller Plane (Management and Control) Considerations
Considerations . . . . . . . . . . . . . . . . . . . . . . . 15 4.1. DetNet Controller Plane Requirements
4.1. DetNet Controller Plane Requirements . . . . . . . . . . 15 4.2. Generic Controller Plane Considerations
4.2. Generic Controller Plane Considerations . . . . . . . . . 17 4.2.1. Flow Aggregation Control
4.2.1. Flow Aggregation Control . . . . . . . . . . . . . . 17 4.2.2. Explicit Routes
4.2.2. Explicit Routes . . . . . . . . . . . . . . . . . . . 18 4.2.3. Contention Loss and Jitter Reduction
4.2.3. Contention Loss and Jitter Reduction . . . . . . . . 19 4.2.4. Bidirectional Traffic
4.2.4. Bidirectional Traffic . . . . . . . . . . . . . . . . 19 4.3. Packet Replication, Elimination, and Ordering Functions
4.3. Packet Replication, Elimination, and Ordering (PREOF) . . 20 (PREOF)
5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 5. Security Considerations
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 6. IANA Considerations
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 7. References
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 7.1. Normative References
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 7.2. Informative References
9.1. Normative References . . . . . . . . . . . . . . . . . . 22 Acknowledgements
9.2. Informative References . . . . . . . . . . . . . . . . . 22 Contributors
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses
1. Introduction 1. Introduction
DetNet (Deterministic Networking) provides a capability to carry DetNet (Deterministic Networking) provides the ability to carry
specified unicast or multicast data flows for real-time applications specified unicast or multicast data flows for real-time applications
with extremely low packet loss rates and assured maximum end-to-end with extremely low packet loss rates and assured maximum end-to-end
delivery latency. A description of the general background and delivery latency. A description of the general background and
concepts of DetNet can be found in [RFC8655]. concepts of DetNet can be found in [RFC8655].
This document describes the concepts needed by any DetNet data plane This document describes the concepts needed by any DetNet data plane
specification (i.e., the DetNet-specific use of packet header fields) specification (i.e., the DetNet-specific use of packet header fields)
and provides considerations for any corresponding implementation. It and provides considerations for any corresponding implementation. It
covers the building blocks that provide the DetNet service, the covers the building blocks that provide the DetNet service, the
DetNet service sub-layer and the DetNet forwarding sub-layer DetNet service sub-layer, and the DetNet forwarding sub-layer
functions as described in the DetNet Architecture. functions as described in the DetNet architecture [RFC8655].
The DetNet Architecture models the DetNet related data plane The DetNet architecture models the DetNet-related data plane
functions as being decomposed into two sub-layers: a service sub- functions as being decomposed into two sub-layers: a service
layer and a forwarding sub-layer. The service sub-layer is used to sub-layer and a forwarding sub-layer. The service sub-layer is used
provide DetNet service protection and reordering. The forwarding to provide DetNet service protection and reordering. The forwarding
sub-layer leverages Traffic Engineering mechanisms and provides sub-layer leverages traffic engineering mechanisms and provides
congestion protection (low loss, assured latency, and limited out-of- congestion protection (low loss, assured latency, and limited out-of-
order delivery). A particular forwarding sub-layer may have order delivery). A particular forwarding sub-layer may have
capabilities that are not available on other forwarding-sub layers. capabilities that are not available on other forwarding sub-layers.
DetNet makes use of the existing forwarding sub-layers with their DetNet makes use of the existing forwarding sub-layers with their
respective capabilities and does not require 1:1 equivalence between respective capabilities and does not require 1:1 equivalence between
different forwarding sub-layer capabilities. different forwarding sub-layer capabilities.
As part of the service sub-layer functions, this document describes As part of the service sub-layer functions, this document describes
typical DetNet node data plane operation. It describes the typical DetNet node data plane operation. It describes the
functionality and operation of the Packet Replication (PRF), Packet functionality and operation of the Packet Replication Function (PRF),
Elimination (PEF), and the Packet Ordering (POF) functions within the the Packet Elimination Function (PEF), and the Packet Ordering
service sub-layer. Furthermore, it describes the forwarding sub- Function (POF) within the service sub-layer. Furthermore, it
layer. describes the forwarding sub-layer.
As defined in [RFC8655], DetNet flows may be carried over network As defined in [RFC8655], DetNet flows may be carried over network
technologies that can provide the DetNet required service technologies that can provide service characteristics required by
characteristics. For example, DetNet MPLS flows can be carried over DetNet. For example, DetNet MPLS flows can be carried over IEEE
IEEE 802.1 Time Sensitive Network (TSN) [IEEE802.1TSNTG] sub- 802.1 Time-Sensitive Networking (TSN) sub-networks [IEEE802.1TSNTG].
networks. However, IEEE 802.1 TSN support is not required in DetNet. However, IEEE 802.1 TSN support is not required in DetNet. TSN frame
TSN frame preemption is an example of a forwarding layer capability preemption is an example of a forwarding layer capability that is
that is typically not replicated in other forwarding technologies. typically not replicated in other forwarding technologies. Most of
Most of DetNet benefits can be gained by running over a data link DetNet's benefits can be gained by running over a data-link layer
layer that has not been specifically enhanced to support all TSN that has not been specifically enhanced to support all TSN
capabilities but for such networks and traffic mixes delay and jitter capabilities, but for such networks and traffic mixes, delay and
performance may vary due to the forwarding sub-layer intrinsic jitter performance may vary due to the forwarding sub-layer's
properties. intrinsic properties.
Different application flows, such as Ethernet or IP, can be mapped on Different application flows, such as Ethernet or IP, can be mapped on
top of DetNet. DetNet can optionally reuse header information top of DetNet. DetNet can optionally reuse header information
provided by, or shared with, applications. An example of shared provided by, or shared with, applications. An example of shared
header fields can be found in [I-D.ietf-detnet-ip]. header fields can be found in [RFC8939].
This document also covers basic concepts related to the controller This document also covers basic concepts related to the Controller
plane and Operations, Administration, and Maintenance (OAM). Data Plane and Operations, Administration, and Maintenance (OAM). Data
plane OAM specifics are out of scope for this document. plane OAM specifics are out of scope for this document.
2. Terminology 2. Terminology
2.1. Terms Used in This Document 2.1. Terms Used in This Document
This document uses the terminology established in the DetNet This document uses the terminology established in the DetNet
architecture [RFC8655], and the reader is assumed to be familiar with architecture [RFC8655], and it is assumed that the reader is familiar
that document and its terminology. with that document and its terminology.
2.2. Abbreviations 2.2. Abbreviations
The following abbreviations are used in this document: The following abbreviations are used in this document:
BGP Border Gateway Protocol. BGP Border Gateway Protocol
d-CW DetNet Control Word.
DetNet Deterministic Networking.
DN DetNet.
GMPLS Generalized Multiprotocol Label Switching.
GRE Generic Routing Encapsulation.
IPSec IP Security.
L2 Layer 2.
LSP Label Switched Path.
MPLS Multiprotocol Label Switching.
OAM Operations, Administration, and Maintenance.
PCEP Path Computation Element Communication Protocol.
PEF Packet Elimination Function.
PRF Packet Replication Function.
PREOF Packet Replication, Elimination and Ordering Functions.
POF Packet Ordering Function.
PSN Packet Switched Network.
QoS Quality of Service.
S-Label DetNet "service" label.
TDM Time-Division Multiplexing.
TSN Time-Sensitive Network.
YANG Yet Another Next Generation.
3. DetNet Data Plane Overview CoS Class of Service
This document describes how application flows, or app-flows, are d-CW DetNet Control Word
carried over DetNet networks. The DetNet Architecture [RFC8655]
models the DetNet-related data plane functions as decomposed into two DetNet Deterministic Networking
sub-layers: a service sub-layer and a forwarding sub-layer.
DN DetNet
GMPLS Generalized Multiprotocol Label Switching
GRE Generic Routing Encapsulation
IPsec IP Security
L2 Layer 2
LSP Label Switched Path
MPLS Multiprotocol Label Switching
OAM Operations, Administration, and Maintenance
PCEP Path Computation Element Communication Protocol
PEF Packet Elimination Function
POF Packet Ordering Function
PREOF Packet Replication, Elimination, and Ordering Functions
PRF Packet Replication Function
PSN Packet Switched Network
QoS Quality of Service
S-Label DetNet "service" label
TDM Time-Division Multiplexing
TSN Time-Sensitive Networking
YANG Yet Another Next Generation
3. Overview of the DetNet Data Plane
This document describes how application flows, or App-flows
[RFC8655], are carried over DetNet networks. The DetNet architecture
[RFC8655] models the DetNet-related data plane functions as
decomposed into two sub-layers: a service sub-layer and a forwarding
sub-layer.
Figure 1, reproduced from [RFC8655], shows a logical DetNet service Figure 1, reproduced from [RFC8655], shows a logical DetNet service
with the two sub-layers. with the two sub-layers.
| packets going | ^ packets coming ^ | packets going | ^ packets coming ^
v down the stack v | up the stack | v down the stack v | up the stack |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Source | | Destination | | Source | | Destination |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Service sub-layer: | | Service sub-layer: | | Service sub-layer: | | Service sub-layer: |
skipping to change at page 5, line 24 skipping to change at line 235
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Forwarding sub-layer: | | Forwarding sub-layer: | | Forwarding sub-layer: | | Forwarding sub-layer: |
| Resource allocation | | Resource allocation | | Resource allocation | | Resource allocation |
| Explicit routes | | Explicit routes | | Explicit routes | | Explicit routes |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
| Lower layers | | Lower layers | | Lower layers | | Lower layers |
+-----------------------+ +-----------------------+ +-----------------------+ +-----------------------+
v ^ v ^
\_________________________/ \_________________________/
Figure 1: DetNet data plane protocol stack Figure 1: DetNet Data Plane Protocol Stack
The DetNet forwarding sub-layer may be directly provided by the The DetNet forwarding sub-layer may be directly provided by the
DetNet service sub-layer, for example by IP tunnels or MPLS. DetNet service sub-layer -- for example, by IP tunnels or MPLS.
Alternatively, an overlay approach may be used in which the packet is Alternatively, an overlay approach may be used in which the packet is
natively carried between key nodes within the DetNet network (say natively carried between key nodes within the DetNet network (say,
between PREOF nodes) and a sub-layer is used to provide the between PREOF nodes), and a sub-layer is used to provide the
information needed to reach the next hop in the overlay. information needed to reach the next hop in the overlay.
The forwarding sub-layer provides the QoS related functions needed by The forwarding sub-layer provides the QoS-related functions needed by
the DetNet flow. It may do this directly through the use of queuing the DetNet flow. It may do this directly through the use of queuing
techniques and traffic engineering methods, or it may do this through techniques and traffic engineering methods, or it may do this through
the assistance of its underlying connectivity. For example it may the assistance of its underlying connectivity. For example, it may
call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN
[IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for [IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for
packet queuing, as well as reservation and allocation of bandwidth packet queuing, as well as reservation and allocation of bandwidth
capacity resources. capacity resources.
The service sub-layer provides additional support beyond the The service sub-layer provides additional support beyond the
connectivity function of the forwarding sub-layer. See Section 4.3 connectivity function of the forwarding sub-layer. See Section 4.3
as an example for Packet Replication, Elimination, and Ordering regarding PREOF. The POF uses sequence numbers added to packets,
functions. The ordering function (POF) uses sequence numbers added enabling a range of packet order protection from simple ordering and
to packets enabling a range of packet order protection from simple dropping out-of-order packets to more complex reordering of a fixed
ordering and dropping out-of-order packets to more complex reordering number of out-of-order, minimally delayed packets. Reordering
of a fixed number of out-of-order, minimally delayed packets. requires buffer resources and has an impact on the delay and jitter
Reordering requires buffer resources and has impact on the delay and of packets in the DetNet flow.
jitter of packets in the DetNet flow.
The method of instantiating each of the layers is specific to the The method of instantiating each of the layers is specific to the
particular DetNet data plane method, and more than one approach may particular DetNet data plane method, and more than one approach may
be applicable to a given network type. be applicable to a given network type.
3.1. Data Plane Characteristics 3.1. Data Plane Characteristics
There are two major characteristics to the data plane: the technology The data plane has two major characteristics: the technology and the
and the encapsulation, as discussed below. encapsulation, as discussed below.
3.1.1. Data Plane Technology 3.1.1. Data Plane Technology
The DetNet data plane is provided by the DetNet service and The DetNet data plane is provided by the DetNet service and
forwarding sub layers. The DetNet service sub-layer generally forwarding sub-layers. The DetNet service sub-layer generally
provides its functions for the DetNet application flows by using or provides its functions for the DetNet application flows by using or
applying existing standardized headers and/or encapsulations. The applying existing standardized headers and/or encapsulations. The
Detnet forwarding sub-layer may provide capabilities leveraging that DetNet forwarding sub-layer may provide capabilities leveraging that
same header or encapsulation technology (e.g., DN IP or DN MPLS) or same header or encapsulation technology (e.g., DN IP or DN MPLS), or
it may be achieved by other technologies as shown in Figure 2. it may be achieved via other technologies, as shown in Figure 2
DetNet is currently defined for operation over packet-switched (IP) below. DetNet is currently defined for operation over packet-
networks or label-switched (MPLS) networks. switched (IP) networks or label-switched (MPLS) networks.
3.1.2. Encapsulation 3.1.2. Encapsulation
DetNet encodes specific flow attributes (flow identity and sequence DetNet encodes specific flow attributes (flow identity and sequence
number) in packets. For example, in DetNet IP, zero encapsulation is number) in packets. For example, in DetNet IP, zero encapsulation is
used and no sequence number is available, and in DetNet MPLS, DetNet- used, and no sequence number is available; in DetNet MPLS, DetNet-
specific information may be added explicitly to the packets in the specific information may be added explicitly to the packets in the
form of S-label and d-CW [I-D.ietf-detnet-mpls] . form of an S-Label and a d-CW [DetNet-MPLS].
The encapsulation of a DetNet flow allows it to be sent over a data The encapsulation of a DetNet flow allows it to be sent over a data
plane technology other than its native type. DetNet uses header plane technology other than its native type. DetNet uses header
information to perform traffic classification, i.e., identify DetNet information to perform traffic classification, i.e., identify DetNet
flows, and provide DetNet service and forwarding functions. As flows, and provide DetNet service and forwarding functions. As
mentioned above, DetNet may add headers, as is the case for DN MPLS, mentioned above, DetNet may add headers, as is the case for DN MPLS,
or may use headers that are already present, as is the case in DN IP. or may use headers that are already present, as is the case for DN
Figure 2 illustrates some relationships between the components. IP. Figure 2 illustrates some relationships between the components.
+-----+ +-----+
| TSN | | TSN |
+-------+ +-+-----+-+ +-------+ +-+-----+-+
| DN IP | | DN MPLS | | DN IP | | DN MPLS |
+--+--+----+----+ +-+---+-----+-+ +--+--+----+----+ +-+---+-----+-+
| TSN | DN MPLS | | TSN | DN IP | | TSN | DN MPLS | | TSN | DN IP |
+-----+---------+ +-----+-------+ +-----+---------+ +-----+-------+
Figure 2: DetNet Service Examples Figure 2: DetNet Service Examples
The use of encapsulation is also required if additional information The use of encapsulation is also required if additional information
(metadata) is needed by the DetNet data plane and there is either no (metadata) is needed by the DetNet data plane and either (1) there is
ability to include it in the client data packet, or the specification no ability to include it in the client data packet or (2) the
of the client data plane does not permit the modification of the specification of the client data plane does not permit the
packet to include additional data. An example of such metadata is modification of the packet to include additional data. An example of
the inclusion of a sequence number required by the PREOF function. such metadata is the inclusion of a sequence number required by
PREOF.
Encapsulation may also be used to carry or aggregate flows for Encapsulation may also be used to carry or aggregate flows for
equipment with limited DetNet capability. equipment with limited DetNet capability.
3.2. DetNet-specific Metadata 3.2. DetNet-Specific Metadata
The DetNet data plane can provide or carry the following metadata: The DetNet data plane can provide or carry the following metadata:
1. Flow-ID 1. Flow-ID
2. Sequence Number 2. Sequence number
The DetNet data plane framework supports a Flow-ID (for The DetNet data plane framework supports a Flow-ID (for
identification of the flow or aggregate flow) and/or a Sequence identification of the flow or aggregate flow) and/or a sequence
Number (for PREOF) for each DetNet flow. The Flow-ID is used by both number (for PREOF) for each DetNet flow. The Flow-ID is used by both
the service and forwarding sub-layers, but the sequence number is the service and forwarding sub-layers, but the sequence number is
only used by the service layer. Metadata can also be used for OAM only used by the service layer. Metadata can also be used for OAM
indications and instrumentation of DetNet data plane operation. indications and instrumentation of DetNet data plane operation.
Metadata inclusion can be implicit or explicit. Explicit inclusions Metadata inclusion can be implicit or explicit. Explicit inclusions
involve a dedicated header field that is used to include metadata in involve a dedicated header field that is used to include metadata in
a DetNet packet. In the implicit method, part of an already existing a DetNet packet. In the implicit method, part of an already-existing
header field is used to encode the metadata. header field is used to encode the metadata.
Explicit inclusion of metadata is possible through the use of IP Explicit inclusion of metadata is possible through the use of IP
options or IP extension headers. New IP options are almost options or IP extension headers. New IP options are almost
impossible to get standardized or to deploy in an operational network impossible to get standardized or to deploy in an operational network
and will not be discussed further in this text. IPv6 extensions and will not be discussed further in this text. IPv6 extension
headers are finding popularity in current IPv6 development work, headers are finding popularity in current IPv6 development work,
particularly in connection with Segment Routing of IPv6 (SRv6) and IP particularly in connection with Segment Routing of IPv6 (SRv6) and IP
OAM. The design of a new IPv6 extension header or the modification OAM. The design of a new IPv6 extension header or the modification
of an existing one is a technique available in the tool box of the of an existing one is a technique available in the toolbox of the
DetNet IP data plane designer. DetNet IP data plane designer.
Explicit inclusion of metadata in an IP packet is also possible Explicit inclusion of metadata in an IP packet is also possible
through the inclusion of an MPLS label stack and the MPLS DetNet through the inclusion of an MPLS label stack and the MPLS d-CW, using
Control Word using one of the methods for carrying MPLS over IP one of the methods for carrying MPLS over IP
[I-D.ietf-detnet-mpls-over-udp-ip]. This is described in more detail [DetNet-MPLS-over-UDP-IP]. This is described in more detail in
in Section 3.5.5. Section 3.5.5.
Implicit metadata in IP can be included through the use of the Implicit metadata in IP can be included through the use of the
network programming paradigm network programming paradigm [SRv6-Network-Prog], in which the suffix
of an IPv6 address is used to encode additional information for use
[I-D.ietf-spring-srv6-network-programming] in which the suffix of an by the network of the receiving host.
IPv6 address is used to encode additional information for use by the
network of the receiving host.
An MPLS example of explicit metadata is the sequence number used by An MPLS example of explicit metadata is the sequence number used by
the PREOF function, or even the case where all the essential PREOF, or even the case where all the essential information is
information being included into the DetNet over MPLS label stack (the included in the DetNet-over-MPLS label stack (the d-CW and the DetNet
DetNet Control Word and the DetNet Service label). S-Label).
3.3. DetNet IP Data Plane 3.3. DetNet IP Data Plane
An IP data plane may operate natively or through the use of an An IP data plane may operate natively or through the use of an
encapsulation. Many types of IP encapsulation can satisfy DetNet encapsulation. Many types of IP encapsulation can satisfy DetNet
requirements and it is anticipated that more than one encapsulation requirements, and it is anticipated that more than one encapsulation
may be deployed, for example GRE, IPSec. may be deployed -- for example, GRE, IPsec.
One method of operating an IP DetNet data plane without encapsulation One method of operating an IP DetNet data plane without encapsulation
is to use "6-tuple" based flow identification, where "6-tuple" refers is to use 6-tuple-based flow identification, where "6-tuple" refers
to information carried in IP and higher layer protocol headers. to information carried in IP-layer and higher-layer protocol headers.
General background on the use of IP headers, and "6-tuples", to General background on the use of IP headers and 6-tuples to identify
identify flows and support Quality of Service (QoS) can be found in flows and support QoS can be found in [RFC3670]. The extra field in
[RFC3670]. [RFC7657] provides useful background on differentiated the 6-tuple is the DSCP field in the packet. [RFC7657] provides
services (DiffServ) and "tuple" based flow identification. DetNet useful background on differentiated services (Diffserv) and tuple-
flow aggregation may be enabled via the use of wildcards, masks, based flow identification. DetNet flow aggregation may be enabled
prefixes and ranges. The operation of this method is described in via the use of wildcards, masks, prefixes, and ranges. The operation
detail in [I-D.ietf-detnet-ip]. of this method is described in detail in [RFC8939].
The DetNet forwarding plane may use explicit route capabilities and The DetNet forwarding plane may use explicit route capabilities and
traffic engineering capabilities to provide a forwarding sub-layer traffic engineering capabilities to provide a forwarding sub-layer
that is responsible for providing resource allocation and explicit that is responsible for providing resource allocation and explicit
routes. It is possible to include such information in a native IP routes. It is possible to include such information in a native IP
packet explicitly, or implicitly. packet either explicitly or implicitly.
3.4. DetNet MPLS Data Plane 3.4. DetNet MPLS Data Plane
MPLS provides a forwarding sub-layer for traffic over implicit and MPLS provides a forwarding sub-layer for traffic over implicit and
explicit paths to the point in the network where the next DetNet explicit paths to the point in the network where the next DetNet
service sub-layer action needs to take place. It does this through service sub-layer action needs to take place. It does this through
the use of a stack of one or more labels with various forwarding the use of a stack of one or more labels with various forwarding
semantics. semantics.
MPLS also provides the ability to identify a service instance that is MPLS also provides the ability to identify a service instance that is
used to process the packet through the use of a label that maps the used to process the packet through the use of a label that maps the
packet to a service instance. packet to a service instance.
In cases where metadata is needed to process an MPLS encapsulated In cases where metadata is needed to process an MPLS-encapsulated
packet at the service sub-layer, the d-CW [I-D.ietf-detnet-mpls], packet at the service sub-layer, the d-CW [DetNet-MPLS] can be used.
[RFC4385] can be used. Although such d-CWs are frequently 32 bits Although such d-CWs are frequently 32 bits long, there is no
long, there is no architectural constraint on the size of this architectural constraint on the size of this structure -- only the
structure, only the requirement that it is fully understood by all requirement that it be fully understood by all parties operating on
parties operating on it in the DetNet service sub-layer. The it in the DetNet service sub-layer. The operation of this method is
operation of this method is described in detail in described in detail in [DetNet-MPLS].
[I-D.ietf-detnet-mpls].
3.5. Further DetNet Data Plane Considerations 3.5. Further DetNet Data Plane Considerations
This section provides informative considerations related to providing This section provides informative considerations related to providing
DetNet service to flows which are identified based on their header DetNet service to flows that are identified based on their header
information. information.
3.5.1. Per Flow Related Functions 3.5.1. Functions Provided on a Per-Flow Basis
At a high level, the following functions are provided on a per flow At a high level, the following functions are provided on a per-flow
basis. basis.
3.5.1.1. Reservation and Allocation of resources 3.5.1.1. Reservation and Allocation of Resources
Resources might be reserved in order to make them available for Resources might be reserved in order to make them available for
allocation to specific DetNet flows. This can eliminate packet allocation to specific DetNet flows. This can eliminate packet
contention and packet loss for DetNet traffic. This also can reduce contention and packet loss for DetNet traffic. This also can reduce
jitter for DetNet traffic. Resources allocated to a DetNet flow jitter for DetNet traffic. Resources allocated to a DetNet flow
protect it from other traffic flows. On the other hand, DetNet flows protect it from other traffic flows. On the other hand, it is
are assumed to behave with respect to the reserved traffic profile. assumed that DetNet flows behave in accordance with the reserved
It must be possible to detect misbehaving DetNet flows and to ensure traffic profile. It must be possible to detect misbehaving DetNet
that they do not compromise QoS of other flows. Queuing, policing, flows and to ensure that they do not compromise QoS of other flows.
and shaping policies can be used to ensure that the allocation of Queuing, policing, and shaping policies can be used to ensure that
resources reserved for DetNet is met. the allocation of resources reserved for DetNet is met.
3.5.1.2. Explicit routes 3.5.1.2. Explicit Routes
A flow can be routed over a specific, pre-computed path. This allows A flow can be routed over a specific, precomputed path. This allows
control of the network delay by steering the packet with the ability control of network delay by steering the packet with the ability to
to influence the physical path. Explicit routes complement influence the physical path. Explicit routes complement reservation
reservation by ensuring that a consistent path can be associated with by ensuring that a consistent path can be associated with its
its resources for the duration of that path. Coupled with the resources for the duration of that path. Coupled with the traffic
traffic mechanism, this limits misordering and bounds latency. mechanism, this limits misordering and bounds latency. Explicit
Explicit route computation can encompass a wide set of constraints route computation can encompass a wide set of constraints and can
and can optimize the path for a certain characteristic, e.g., highest optimize the path for a certain characteristic, e.g., highest
bandwidth or lowest jitter. In these cases the "best" path for any bandwidth or lowest jitter. In these cases, the "best" path for any
set of characteristics may not be a shortest path. The selection of set of characteristics may not be a shortest path. The selection of
path can take into account multiple network metrics. Some of these the path can take into account multiple network metrics. Some of
metrics are measured and distributed by the routing system as traffic these metrics are measured and distributed by the routing system as
engineering metrics. traffic engineering metrics.
3.5.1.3. Service protection 3.5.1.3. Service Protection
Service protection involves use of multiple packet streams using Service protection involves the use of multiple packet streams using
multiple paths, for example 1+1 or 1:1 linear protection. For multiple paths -- for example, 1+1 or 1:1 linear protection. For
DetNet, this primarily relates to packet replication and elimination DetNet, this primarily relates to packet replication and elimination
capabilities. MPLS offers a number of protection schemes. MPLS capabilities. MPLS offers a number of protection schemes. MPLS
hitless protection can be used to switch traffic to an already hitless protection can be used to switch traffic to an already-
established path in order to restore delivery rapidly after a established path in order to restore delivery rapidly after a
failure. Path changes, even in the case of failure recovery, can failure. Path changes, even in the case of failure recovery, can
lead to the out of order delivery of data requiring packet ordering lead to the out-of-order delivery of data requiring POFs either
functions either within the DetNet service or at a high layer in the within the DetNet service or at a high layer in the application
application traffic. Establishment of new paths after a failure is traffic. Establishment of new paths after a failure is out of scope
out of scope for DetNet services. for DetNet services.
3.5.1.4. Network Coding 3.5.1.4. Network Coding
Network Coding [nwcrg], not to be confused with network programming, Network Coding [nwcrg], not to be confused with network programming,
comprises several techniques where multiple data flows are encoded. comprises several techniques where multiple data flows are encoded.
These resulting flows can then be sent on different paths. The These resulting flows can then be sent on different paths. The
encoding operation can combine flows and error recovery information. encoding operation can combine flows and error recovery information.
When the encoded flows are decoded and recombined the original flows When the encoded flows are decoded and recombined, the original flows
can be recovered. Note that Network coding uses an alternative to can be recovered. Note that Network Coding uses an alternative to
packet by packet PREOF. Therefore, for certain network topologies packet-by-packet PREOF. Therefore, for certain network topologies
and traffic loads, Network Coding can be used to improve a network's and traffic loads, Network Coding can be used to improve a network's
throughput, efficiency, latency, and scalability, as well as throughput, efficiency, latency, and scalability, as well as
resilience to partition, attacks, and eavesdropping, as compared to resilience to partition, attacks, and eavesdropping, as compared to
traditional methods. DetNet could use Network coding as an traditional methods. DetNet could use Network Coding as an
alternative to other protection means. Network coding is often alternative to other means of protection. Network Coding is often
applied in wireless networks and is being explored for other network applied in wireless networks and is being explored for other network
types. types.
3.5.1.5. Load sharing 3.5.1.5. Load-Sharing
Use of packet-by-packet load sharing of the same DetNet flow over The use of packet-by-packet load-sharing of the same DetNet flow over
multiple paths is not recommended except for the cases listed above multiple paths is not recommended, except for the cases listed above
where PREOF is utilized to improve protection of traffic and maintain where PREOF are utilized to improve protection of traffic and
order. Packet-by-packet load sharing, e.g., via ECMP or UCMP, maintain order. Packet-by-packet load-sharing, e.g., via Equal-Cost
impacts ordering and possibly jitter. Multipath (ECMP) or Unequal-Cost Multipath (UCMP), impacts ordering
and, possibly, jitter.
3.5.1.6. Troubleshooting 3.5.1.6. Troubleshooting
Detnet leverages many different forwarding sub-layers, each of which DetNet leverages many different forwarding sub-layers, each of which
supports various tools to troubleshoot connectivity, for example supports various tools to troubleshoot connectivity -- for example,
identification of misbehaving flows. The DetNet Service layer can identification of misbehaving flows. The DetNet service layer can
leverage existing mechanisms to troubleshoot or monitor flows, such leverage existing mechanisms to troubleshoot or monitor flows, such
as those in use by IP and MPLS networks. At the Application layer a as those in use by IP and MPLS networks. At the Application layer, a
client of a DetNet service can use existing techniques to detect and client of a DetNet service can use existing techniques to detect and
monitor delay and loss. monitor delay and loss.
3.5.1.7. Flow recognition for analytics 3.5.1.7. Flow Recognition for Analytics
Network analytics can be inherited from the technologies of the Network analytics can be inherited from the technologies of the
Service and Forwarding sub-layers. At the DetNet service edge, service and forwarding sub-layers. At the DetNet service edge,
packet and bit counters (e.g. sent, received, dropped, and out-of- packet and bit counters (e.g., sent, received, dropped, out of
sequence) can be maintained. sequence) can be maintained.
3.5.1.8. Correlation of events with flows 3.5.1.8. Correlation of Events with Flows
The provider of a DetNet service may provide other capabilities to The provider of a DetNet service may provide other capabilities to
monitor flows, such as more detailed loss statistics and time monitor flows, such as more detailed loss statistics and timestamping
stamping of events. The details of these capabilities are out of of events. Details regarding these capabilities are out of scope for
scope for this document. this document.
3.5.2. Service Protection 3.5.2. Service Protection
Service protection allows DetNet services to increase reliability and Service protection allows DetNet services to increase reliability and
maintain a DetNet Service Assurance in the case of network congestion maintain a desired level of service assurance in the case of network
or network failure. Detnet relies on the underlying technology congestion or network failure. DetNet relies on the underlying
capabilities for various protection schemes. Protection schemes technology capabilities for various protection schemes. Protection
enable partial or complete coverage of the network paths and active schemes enable partial or complete coverage of the network paths and
protection with combinations of PRF, PEF, and POF. active protection with combinations of the PRF, PEF, and POF.
3.5.2.1. Linear Service Protection 3.5.2.1. Linear Service Protection
An example DetNet MPLS network fragment and packet flow is An example DetNet MPLS network fragment and its packet flow are
illustrated in Figure 3. illustrated in Figure 3.
1 1.1 1.1 1.2.1 1.2.1 1.2.2 1 1.1 1.1 1.2.1 1.2.1 1.2.2
CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 CE1----EN1--------R1-------R2-------R3--------EN2-----CE2
\ 1.2.1 / / \ 1.2.1 / /
\1.2 /-----+ / \1.2 /------+ /
+------R4------------------------+ +------R4------------------------+
1.2.2 1.2.2
Figure 3: Example Packet Flow in DetNet Protected Network Figure 3: Example of Packet Flow Protected by DetNet
In Figure 3 the numbers are used to identify the instance of a In Figure 3, the numbers are used to identify the instance of a
packet. Packet 1 is the original packet, and packets 1.1, and 1.2 packet. Packet 1 is the original packet. Packets 1.1 and 1.2 are
are two first generation copies of packet 1. Packet 1.2.1 is a two first-generation copies of packet 1, packet 1.2.1 is a second-
second generation copy of packet 1.2, etc. Note that these numbers generation copy of packet 1.2, and so on. Note that these numbers
never appear in the packet, and are not to be confused with sequence never appear in the packet and are not to be confused with sequence
numbers, labels or any other identifier that appears in the packet. numbers, labels, or any other identifiers that appear in the packet.
They simply indicate the generation number of the original packet so They simply indicate the generation number of the original packet so
that its passage through the network fragment can be identified to that its passage through the network fragment can be identified for
the reader. the reader.
Customer Equipment CE1 sends a packet into the DetNet enabled Customer Equipment device CE1 sends a packet into the DetNet-enabled
network. This is packet (1). Edge Node EN1 encapsulates the packet network. This is packet 1. Edge Node EN1 encapsulates the packet as
as a DetNet packet and sends it to Relay node R1 (packet 1.1). EN1 a DetNet packet and sends it to Relay Node R1 (packet 1.1). EN1
makes a copy of the packet (1.2), encapsulates it and sends this copy makes a copy of the packet (1.2), encapsulates it, and sends this
to Relay node R4. copy to Relay Node R4.
Note that along the path from EN1 to R1 there may be zero or more Note that R1 may be directly attached to EN1, or there may be one or
nodes which, for clarity, are not shown. The same is true for any more nodes on the path that, for clarity, are not shown in Figure 3.
other path between two DetNet entities shown in Figure 3. The same holds true for any other path between two DetNet entities as
shown in the figure.
Relay node R4 has been configured to send one copy of the packet to Relay Node R4 has been configured to send one copy of the packet to
Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet
1.2.2). 1.2.2).
R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, R2 receives packet copy 1.2.1 before packet copy 1.1 arrives and,
having been configured to perform packet elimination on this DetNet having been configured to perform packet elimination on this DetNet
flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of
no further use and so is discarded by R2. no further use and so is discarded by R2.
Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives
packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips packet copy 1.2.1 from R2 via Relay Node R3. EN2 therefore strips
any DetNet encapsulation from packet copy 1.2.2 and forwards the any DetNet encapsulation from packet copy 1.2.2 and forwards the
packet to CE2. When EN2 receives the later packet copy 1.2.1 this is packet to CE2. When EN2 receives packet copy 1.2.1 later on, the
discarded. copy is discarded.
The above is of course illustrative of many network scenarios that The above is of course illustrative of many network scenarios that
can be configured. can be configured.
This example also illustrates 1:1 protection scheme meaning there is This example also illustrates a 1:1 protection scheme, meaning there
traffic over each segment of the end-to-end path. Local DetNet relay is traffic over each segment of the end-to-end path. Local DetNet
nodes determine which packets are eliminated and which packets are relay nodes determine which packets are eliminated and which packets
forwarded. A 1+1 scheme where only one path is used for traffic at a are forwarded. A 1+1 scheme where only one path is used for traffic
time could use the same topology. In this case there is no PRF at a time could use the same topology. In this case, there is no
function and traffic is switched upon detection of failure. An OAM PRF, and traffic is switched upon detection of failure. An OAM
scheme that monitors the paths to detect the loss of path or traffic scheme that monitors the paths to detect the loss of a path or
is required to initiate the switch. A POF may still be used in this traffic is required to initiate the switch. A POF may still be used
case to prevent misordering of packets. In both cases the protection in this case to prevent misordering of packets. In both cases, the
paths are established and maintained for the duration of the DetNet protection paths are established and maintained for the duration of
service. the DetNet service.
3.5.2.2. Path Differential Delay 3.5.2.2. Path Differential Delay
In the preceding example, proper working of duplicate elimination and In the preceding example, proper operation of duplicate elimination
reordering of packets are dependent on the number of out-of-order and the reordering of packets are dependent on the number of out-of-
packets that can be buffered and the delay difference of arriving order packets that can be buffered and the difference in delay of the
packets. DetNet uses flow-specific requirements (e.g., maximum arriving packets. DetNet uses flow-specific requirements (e.g.,
number of out-of-order packets, maximum latency of the flow) for maximum number of out-of-order packets, maximum latency of the flow)
configuration of POF-related buffers. If the differential delay for configuration of POF-related buffers. If the differential delay
between paths is excessively large or there is excessive mis-ordering between paths is excessively large or there is excessive misordering
of the packets, then packets may be dropped instead of being of the packets, then packets may be dropped instead of being
reordered. Likewise, PEF uses the sequence number to identify reordered. Likewise, the PEF uses the sequence number to identify
duplicate packets, and large differential delays combined with high duplicate packets, and large differential delays combined with high
numbers of packets may exceed the ability of the PEF to work numbers of packets may exceed the PEF's ability to work properly.
properly.
3.5.2.3. Ring Service Protection 3.5.2.3. Ring Service Protection
Ring protection may also be supported if the underlying technology Ring protection may also be supported if the underlying technology
supports it. Many of the same concepts apply, however rings are supports it. Many of the same concepts apply; however, rings are
normally 1+1 protection for data efficiency reasons. [RFC8227] is an normally 1+1 protection for data efficiency reasons. [RFC8227]
example of MPLS-TP data plane that supports Ring protection. provides an example of an MPLS Transport Profile (MPLS-TP) data plane
that supports ring protection.
3.5.3. Aggregation Considerations 3.5.3. Aggregation Considerations
The DetNet data plane also allows for the aggregation of DetNet The DetNet data plane also allows for the aggregation of DetNet
flows, which can improve scalability by reducing the per-hop state. flows, which can improve scalability by reducing the per-hop state.
How this is accomplished is data plane or control plane dependent. How this is accomplished is data plane or control plane dependent.
When DetNet flows are aggregated, transit nodes provide service to When DetNet flows are aggregated, transit nodes provide service to
the aggregate and not on a per-DetNet flow basis. When aggregating the aggregate and not on a per-DetNet-flow basis. When aggregating
DetNet flows, the flows should be compatible, i.e., the same or very DetNet flows, the flows should be compatible, i.e., the same or very
similar QoS and CoS characteristics. In this case, nodes performing similar QoS and CoS characteristics. In this case, nodes performing
aggregation will ensure that per-flow service requirements are aggregation will ensure that per-flow service requirements are
achieved. achieved.
If bandwidth reservations are used, the reservation should be the sum If bandwidth reservations are used, the reservation should be the sum
of all the individual reservations; in other words, the reservations of all the individual reservations; in other words, the reservations
should not add up to an over-subscription of bandwidth reservation. should not add up to an oversubscription of bandwidth reservation.
If maximum delay bounds are used, the system should ensure that the If maximum delay bounds are used, the system should ensure that the
aggregate does not exceed the delay bounds of the individual flows. aggregate does not exceed the delay bounds of the individual flows.
When an encapsulation is used, the choice of reserving a maximum When an encapsulation is used, the choice of reserving a maximum
resource level and then tracking the services in the aggregated resource level and then tracking the services in the aggregated
service or adjusting the aggregated resources as the services are service or adjusting the aggregated resources as the services are
added is implementation and technology specific. added is implementation and technology specific.
DetNet flows at edges must be able to handle rejection to an DetNet flows at edges must be able to handle rejection to an
aggregation group due to lack of resources as well as conditions aggregation group due to lack of resources as well as conditions
where requirements are not satisfied. where requirements are not satisfied.
3.5.3.1. IP Aggregation 3.5.3.1. IP Aggregation
IP aggregation has both data plane and controller plane aspects. For IP aggregation has both data plane and Controller Plane aspects. For
the data plane, flows may be aggregated for treatment based on shared the data plane, flows may be aggregated for treatment based on shared
characteristics such as 6-tuple [I-D.ietf-detnet-ip]. Alternatively, characteristics such as 6-tuple [RFC8939]. Alternatively, an IP
an IP encapsulation may be used to tunnel an aggregate number of encapsulation may be used to tunnel an aggregate number of DetNet
DetNet Flows between relay nodes. flows between relay nodes.
3.5.3.2. MPLS Aggregation 3.5.3.2. MPLS Aggregation
MPLS aggregation also has data plane and controller plane aspects. MPLS aggregation also has data plane and Controller Plane aspects.
MPLS flows are often tunneled in a forwarding sub-layer, under the MPLS flows are often tunneled in a forwarding sub-layer, under the
reservation associated with that MPLS tunnel. reservation associated with that MPLS tunnel.
3.5.4. End-System-Specific Considerations 3.5.4. End-System-Specific Considerations
Data-flows requiring DetNet service are generated and terminated on Data flows requiring DetNet service are generated and terminated on
end-systems. Encapsulation depends on the application and its end systems. Encapsulation depends on the application and its
preferences. For example, in a DetNet MPLS domain the sub-layer preferences. For example, in a DetNet MPLS domain, the sub-layer
functions use the d-CWs, S-Labels and F-Labels to provide DetNet functions use the d-CWs, S-Labels, and F-Labels [DetNet-MPLS] to
services. However, an application may exchange further flow related provide DetNet services. However, an application may exchange
parameters (e.g., time-stamp), which are not provided by DetNet further flow-related parameters (e.g., timestamps) that are not
functions. provided by DetNet functions.
As a general rule, DetNet domains are capable of forwarding any As a general rule, DetNet domains are capable of forwarding any
DetNet flows and the DetNet domain does not mandate the end-system or DetNet flows, and the DetNet domain does not mandate the
edge node encapsulation format. Unless there is a proxy of some form encapsulation format for end systems or edge nodes. Unless some form
present, end-systems peer with similar end-systems using the same of proxy is present, end systems peer with similar end systems using
application encapsulation format. For example, as shown in Figure 4, the same application encapsulation format. For example, as shown in
IP applications peer with IP applications and Ethernet applications Figure 4, IP applications peer with IP applications, and Ethernet
peer with Ethernet applications. applications peer with Ethernet applications.
+-----+ +-----+
| X | +-----+ | X | +-----+
+-----+ | X | +-----+ | X |
| Eth | ________ +-----+ | Eth | ________ +-----+
+-----+ _____ / \ | Eth | +-----+ _____ / \ | Eth |
\ / \__/ \___ +-----+ \ / \__/ \___ +-----+
\ / \ / \ / \ /
0======== tunnel-1 ========0_ 0======== tunnel-1 ========0_
| \ | \
\ | \ |
0========= tunnel-2 =========0 0========= tunnel-2 =========0
/ \ __/ \ / \ __/ \
+-----+ \__ DetNet MPLS domain / \ +-----+ \__ DetNet MPLS domain / \
| X | \ __ / +-----+ | X | \ __ / +-----+
+-----+ \_______/ \_____/ | X | +-----+ \_______/ \_____/ | X |
| IP | +-----+ | IP | +-----+
+-----+ | IP | +-----+ | IP |
+-----+ +-----+
Figure 4: End-Systems and The DetNet MPLS Domain Figure 4: End Systems and the DetNet MPLS Domain
3.5.5. Sub-Network Considerations 3.5.5. Sub-network Considerations
Any of the DetNet service types may be transported by another DetNet Any of the DetNet service types may be transported by another DetNet
service. MPLS nodes may be interconnected by different sub-network service. MPLS nodes may be interconnected by different sub-network
technologies, which may include point-to-point links. Each of these technologies, which may include point-to-point links. Each of these
sub-network technologies needs to provide appropriate service to sub-network technologies needs to provide appropriate service to
DetNet flows. In some cases, e.g., on dedicated point-to-point links DetNet flows. In some cases, e.g., on dedicated point-to-point links
or TDM technologies, all that is required is for a DetNet node to or TDM technologies, all that is required is for a DetNet node to
appropriately queue its output traffic. In other cases, DetNet nodes appropriately queue its output traffic. In other cases, DetNet nodes
will need to map DetNet flows to the flow semantics (i.e., will need to map DetNet flows to the flow semantics (i.e.,
identifiers) and mechanisms used by an underlying sub-network identifiers) and mechanisms used by an underlying sub-network
technology. Figure 5 shows several examples of header formats that technology. Figure 5 shows several examples of sub-network
can be used to carry DetNet MPLS flows over different sub-network encapsulations that can be used to carry DetNet MPLS flows over
technologies. L2 represents a generic layer-2 encapsulation that different sub-network technologies. L2 represents a generic Layer 2
might be used on a point-to-point link. TSN represents the encapsulation that might be used on a point-to-point link. TSN
encapsulation used on an IEEE 802.1 TSN network, as described in represents the encapsulation used on an IEEE 802.1 TSN network, as
[I-D.ietf-detnet-mpls-over-tsn]. UDP/IP represents the encapsulation described in [DetNet-MPLS-over-TSN]. UDP/IP represents the
used on a DetNet IP PSN, as described in encapsulation used on a DetNet IP PSN, as described in
[I-D.ietf-detnet-mpls-over-udp-ip]. [DetNet-MPLS-over-UDP-IP].
+------+ +------+ +------+ +------+ +------+ +------+
App-Flow | X | | X | | X | App-flow | X | | X | | X |
+-----+======+--+======+--+======+-----+ +-----+======+--+======+--+======+-----+
DetNet-MPLS | d-CW | | d-CW | | d-CW | DetNet-MPLS | d-CW | | d-CW | | d-CW |
+------+ +------+ +------+ +------+ +------+ +------+
|Labels| |Labels| |Labels| |Labels| |Labels| |Labels|
+-----+======+--+======+--+======+-----+ +-----+======+--+======+--+======+-----+
Sub-Network | L2 | | TSN | | UDP | Sub-network | L2 | | TSN | | UDP |
+------+ +------+ +------+ +------+ +------+ +------+
| IP | | IP |
+------+ +------+
| L2 | | L2 |
+------+ +------+
Figure 5: Example DetNet MPLS Sub-Network Formats Figure 5: Example DetNet MPLS Encapsulations in Sub-networks
4. Controller Plane (Management and Control) Considerations 4. Controller Plane (Management and Control) Considerations
4.1. DetNet Controller Plane Requirements 4.1. DetNet Controller Plane Requirements
The Controller Plane corresponds to the aggregation of the Control The Controller Plane corresponds to the aggregation of the Control
and Management Planes discussed in [RFC7426] and [RFC8655]. While and Management Planes discussed in [RFC7426] and [RFC8655]. While
more details of any DetNet controller plane are out of the scope of more details regarding any DetNet Controller Plane are out of scope
this document, there are particular considerations and requirements for this document, there are particular considerations and
for such that result from the unique characteristics of the DetNet requirements for the Controller Plane that result from the unique
architecture and data plane as defined herein. characteristics of the DetNet architecture and data plane as defined
herein.
The primary requirements of the DetNet controller plane are that it The primary requirements of the DetNet Controller Plane are that it
must be able to: must be able to:
o Instantiate DetNet flows in a DetNet domain (which may include * Instantiate DetNet flows in a DetNet domain (which may, for
some or all of explicit path determination, link bandwidth example, include some or all of the following: explicit path
reservations, restricting flows to IEEE 802.1 TSN links, node determination, link bandwidth reservations, restricting flows to
buffer and other resource reservations, specification of required IEEE 802.1 TSN links, node buffer and other resource reservations,
queuing disciplines along the path, ability to manage specification of required queuing disciplines along the path,
bidirectional flows, etc.) as needed for a flow. ability to manage bidirectional flows, etc.) as needed for a flow.
o In the case of MPLS, manage DetNet S-Label and F-Label allocation * In the case of MPLS, manage DetNet S-Label and F-Label allocation
and distribution, where the DetNet MPLS encapsulation is in use and distribution. In cases where the DetNet MPLS encapsulation is
see [I-D.ietf-detnet-mpls]. being used, see [DetNet-MPLS].
o Support DetNet flow aggregation. * Support DetNet flow aggregation.
o Advertise static and dynamic node and link resources such as * Advertise static and dynamic node and link resources such as
capabilities and adjacencies to other network nodes (for dynamic capabilities and adjacencies to other network nodes (for dynamic
signaling approaches) or to network controllers (for centralized signaling approaches) or to network controllers (for centralized
approaches). approaches).
o Scale to handle the number of DetNet flows expected in a domain * Scale to handle the number of DetNet flows expected in a domain
(which may require per-flow signaling or provisioning). (which may require per-flow signaling or provisioning).
o Provision flow identification information at each of the nodes * Provision flow identification information at each of the nodes
along the path. Flow identification may differ depending on the along the path. Flow identification may differ, depending on the
location in the network and the DetNet functionality (e.g. transit location in the network and the DetNet functionality (e.g.,
node vs. relay node). transit node vs. relay node).
These requirements, as stated earlier, could be satisfied using These requirements, as stated earlier, could be satisfied using
distributed control protocol signaling (such as RSVP-TE), centralized distributed control protocol signaling (such as RSVP-TE), centralized
network management provisioning mechanisms (such as BGP, PCEP, YANG network management provisioning mechanisms (BGP, PCEP, YANG,
[I-D.ietf-detnet-flow-information-model], etc.) or hybrid [DetNet-Flow-Info], etc.), or hybrid combinations of the two, and
combinations of the two, and could also make use of MPLS-based could also make use of MPLS-based segment routing.
segment routing.
In the abstract, the results of either distributed signaling or In the abstract, the results of either distributed signaling or
centralized provisioning are equivalent from a DetNet data plane centralized provisioning are equivalent from a DetNet data plane
perspective - flows are instantiated, explicit routes are determined, perspective -- flows are instantiated, explicit routes are
resources are reserved, and packets are forwarded through the domain determined, resources are reserved, and packets are forwarded through
using the DetNet data plane. the domain using the DetNet data plane.
However, from a practical and implementation standpoint, controller However, from a practical and implementation standpoint, Controller
plane alternatives are not equivalent at all. Some approaches are Plane alternatives are not equivalent at all. Some approaches are
more scalable than others in terms of signaling load on the network. more scalable than others in terms of signaling load on the network.
Some alternatives can take advantage of global tracking of resources Some alternatives can take advantage of global tracking of resources
in the DetNet domain for better overall network resource in the DetNet domain for better overall network resource
optimization. Some solutions are more resilient than others if link, optimization. Some solutions are more resilient than others if link,
node, or management equipment failures occur. While a detailed node, or management equipment failures occur. While a detailed
analysis of the control plane alternatives is out of the scope of analysis of the control plane alternatives is out of scope for this
this document, the requirements from this document can be used as the document, the requirements from this document can be used as the
basis of a later analysis of the alternatives. basis of a future analysis of the alternatives.
4.2. Generic Controller Plane Considerations 4.2. Generic Controller Plane Considerations
This section covers control plane considerations that are independent This section covers control plane considerations that are independent
of the data plane technology used for DetNet service delivery. of the data plane technology used for DetNet service delivery.
While the management plane and control planes are traditionally While the management plane and the control plane are traditionally
considered separately, from the data plane perspective there is no considered separately, from a data plane perspective, there is no
practical difference based on the origin of flow provisioning practical difference based on the origin of flow-provisioning
information, and the DetNet architecture [RFC8655] refers to these information, and the DetNet architecture [RFC8655] refers to these
collectively as the 'Controller Plane'. This document therefore does collectively as the "Controller Plane". This document therefore does
not distinguish between information provided by distributed control not distinguish between information provided by distributed control
plane protocols, e.g., RSVP-TE [RFC3209] and [RFC3473], or by plane protocols (e.g., RSVP-TE [RFC3209] [RFC3473]) or centralized
centralized network management mechanisms, e.g., RestConf [RFC8040], network management mechanisms (e.g., RESTCONF [RFC8040], YANG
YANG [RFC7950], and the Path Computation Element Communication [RFC7950], PCEP [PCECC]), or any combination thereof. Specific
Protocol (PCEP) [I-D.ietf-pce-pcep-extension-for-pce-controller] or considerations and requirements for the DetNet Controller Plane are
any combination thereof. Specific considerations and requirements discussed in Section 4.1.
for the DetNet Controller Plane are discussed in Section 4.1.
Each respective data plane document also covers the control plane Each respective data plane document also covers the control plane
considerations for that technology. For example, considerations for that technology. For example, [RFC8939] also
[I-D.ietf-detnet-ip] covers IP control plane normative considerations covers IP control plane normative considerations, and [DetNet-MPLS]
and [I-D.ietf-detnet-mpls] covers MPLS control plane normative also covers MPLS control plane normative considerations.
considerations.
4.2.1. Flow Aggregation Control 4.2.1. Flow Aggregation Control
Flow aggregation means that multiple app-flows are served by a single Flow aggregation means that multiple App-flows are served by a single
new DetNet flow. There are many techniques to achieve aggregation. new DetNet flow. There are many techniques to achieve aggregation.
For example, in the case of IP, IP flows that share 6-tuple For example, in the case of IP, IP flows that share 6-tuple
attributes or flow identifiers at the DetNet sub-layer can be attributes or flow identifiers at the DetNet sub-layer can be
grouped. Another example includes aggregation accomplished through grouped. Another example includes aggregation accomplished through
the use of hierarchical LSPs in MPLS and tunnels. the use of hierarchical LSPs in MPLS and tunnels.
Control of aggregation involves a set of procedures listed here. Control of aggregation involves a set of procedures listed here.
Aggregation may use some or all of these capabilities and the order Aggregation may use some or all of these capabilities, and the order
may vary: may vary:
o Traffic engineering resource collection and distribution: Traffic engineering resource collection and distribution:
Available resources are tracked through control plane or
Available resources are tracked through control plane or management plane databases and distributed amongst controllers or
management plane databases and distributed amongst controllers nodes that can manage resources.
or nodes that can manage resources.
o Path computation and resource allocation:
When DetNet services are provisioned or requested, one or more
paths meeting the requirements are selected and the resources
verified and recorded.
o Resource assignment and data plane co-ordination:
The assignment of resources along the path depends on the Path computation and resource allocation:
technology and includes assignment of specific links, When DetNet services are provisioned or requested, one or more
coordination of queueing, and other traffic management paths meeting the requirements are selected and the resources
capabilities such as policing and shaping. verified and recorded.
o Assigned Resource recording and updating: Resource assignment and data plane coordination:
The assignment of resources along the path depends on the
technology and includes assignment of specific links, coordination
of queuing, and other traffic management capabilities such as
policing and shaping.
Depending on the specific technology, the assigned resources Assigned resource recording and updating:
are updated and distributed in the databases, preventing over- Depending on the specific technology, the assigned resources are
subscription. updated and distributed in the databases, preventing
oversubscription.
4.2.2. Explicit Routes 4.2.2. Explicit Routes
Explicit routes are used to ensure that packets are routed through Explicit routes are used to ensure that packets are routed through
the resources that have been reserved for them, and hence provide the the resources that have been reserved for them and hence provide the
DetNet application with the required service. A requirement for the DetNet application with the required service. A requirement for the
DetNet Controller Plane will be the ability to assign a particular DetNet Controller Plane will be the ability to assign a particular
identified DetNet IP flow to a path through the DetNet domain that identified DetNet IP flow to a path through the DetNet domain that
has been assigned the required per-node resources. This provides the has been assigned the required per-node resources. This provides the
appropriate traffic treatment for the flow and also includes appropriate traffic treatment for the flow and also includes
particular links as a part of the path that are able to support the particular links as a part of the path that are able to support the
DetNet flow. For example, by using IEEE 802.1 TSN links (as DetNet flow. For example, by using IEEE 802.1 TSN links (as
discussed in [I-D.ietf-detnet-mpls-over-tsn] ) DetNet parameters can discussed in [DetNet-MPLS-over-TSN]), DetNet parameters can be
be maintained. Further considerations and requirements for the maintained. Further considerations and requirements for the DetNet
DetNet Controller Plane are discussed in Section 4.1. Controller Plane are discussed in Section 4.1.
Whether configuring, calculating and instantiating these routes is a Whether configuring, calculating, and instantiating these routes is a
single-stage or multi-stage process, or in a centralized or single-stage or multi-stage process, or is performed in a centralized
distributed manner, is out of scope of this document. or distributed manner, is out of scope for this document.
There are several approaches that could be used to provide explicit There are several approaches that could be used to provide explicit
routes and resource allocation in the DetNet forwarding sub-layer. routes and resource allocation in the DetNet forwarding sub-layer.
For example: For example:
o The path could be explicitly set up by a controller which * The path could be explicitly set up by a controller that
calculates the path and explicitly configures each node along that calculates the path and explicitly configures each node along that
path with the appropriate forwarding and resource allocation path with the appropriate forwarding and resource allocation
information. information.
o The path could use a distributed control plane such as RSVP * The path could use a distributed control plane such as RSVP
[RFC2205] or RSVP-TE [RFC3473] extended to support DetNet IP [RFC2205] or RSVP-TE [RFC3473] extended to support DetNet IP
flows. flows.
o The path could be implemented using IPv6-based segment routing * The path could be implemented using IPv6-based segment routing
when extended to support resource allocation. when extended to support resource allocation.
See Section 4.1 for further discussion of these alternatives. In See Section 4.1 for further discussion of these alternatives. In
addition, [RFC2386] contains useful background information on QoS- addition, [RFC2386] contains useful background information on QoS-
based routing, and [RFC5575] (being updated by based routing, and [RFC5575] (which will be updated by
[I-D.ietf-idr-rfc5575bis]) discusses a specific mechanism used by BGP [Flow-Spec-Rules]) discusses a specific mechanism used by BGP for
for traffic flow specification and policy-based routing. traffic flow specification and policy-based routing.
4.2.3. Contention Loss and Jitter Reduction 4.2.3. Contention Loss and Jitter Reduction
As discussed in Section 1, this document does not specify the This document does not specify the mechanisms needed to eliminate
mechanisms needed to eliminate packet contention, packet loss or packet contention or packet loss or to reduce jitter for DetNet flows
reduce jitter for DetNet flows at the DetNet forwarding sub-layer. at the DetNet forwarding sub-layer. The ability to manage node and
The ability to manage node and link resources to be able to provide link resources to be able to provide these functions is a necessary
these functions is a necessary part of the DetNet controller plane. part of the DetNet Controller Plane. It is also necessary to be able
It is also necessary to be able to control the required queuing to control the required queuing mechanisms used to provide these
mechanisms used to provide these functions along a flow's path functions along a flow's path through the network. See [RFC8939] and
through the network. See [I-D.ietf-detnet-ip] and Section 4.1 for Section 4.1 for further discussion of these requirements. Some forms
further discussion of these requirements. Some forms of protection of protection may minimize packet loss or change jitter
may minimize packet loss or change jitter characteristics in the characteristics in the cases where packets are reordered when out-of-
cases where packets are reordered when out-of-order packets are order packets are received at the service sub-layer.
received at the service sub-layer.
4.2.4. Bidirectional Traffic 4.2.4. Bidirectional Traffic
In many cases DetNet flows can be considered unidirectional and In many cases, DetNet flows can be considered unidirectional and
independent. However, there are cases where the DetNet service independent. However, there are cases where the DetNet service
requires bidirectional traffic from a DetNet application service requires bidirectional traffic from a DetNet application service
perspective. IP and MPLS typically treat each direction separately perspective. IP and MPLS typically treat each direction separately
and do not force interdependence of each direction. MPLS has and do not force interdependence of each direction. The IETF MPLS
considered bidirectional traffic requirements and the MPLS Working Group has studied bidirectional traffic requirements. The
definitions from [RFC5654] are useful to illustrate terms such as definitions provided in [RFC5654] are useful to illustrate terms such
associated bidirectional flows and co-routed bidirectional flows. as associated bidirectional flows and co-routed bidirectional flows.
MPLS defines a point-to-point associated bidirectional LSP as MPLS defines a point-to-point associated bidirectional LSP as
consisting of two unidirectional point-to-point LSPs, one from A to B consisting of two unidirectional point-to-point LSPs, one from A to B
and the other from B to A, which are regarded as providing a single and the other from B to A, which are regarded as providing a single
logical bidirectional forwarding path. This is analogous to standard logical bidirectional forwarding path. This is analogous to standard
IP routing. MPLS defines a point-to-point co-routed bidirectional IP routing. MPLS defines a point-to-point co-routed bidirectional
LSP as an associated bidirectional LSP which satisfies the additional LSP as an associated bidirectional LSP that satisfies the additional
constraint that its two unidirectional component LSPs follow the same constraint that its two unidirectional component LSPs follow the same
path (in terms of both nodes and links) in both directions. An path (in terms of both nodes and links) in both directions. An
important property of co-routed bidirectional LSPs is that their important property of co-routed bidirectional LSPs is that their
unidirectional component LSPs share fate. In both types of unidirectional component LSPs share fate. In both types of
bidirectional LSPs, resource reservations may differ in each bidirectional LSPs, resource reservations may differ in each
direction. The concepts of associated bidirectional flows and co- direction. The concepts of associated bidirectional flows and
routed bidirectional flows can also be applied to DetNet IP flows. co-routed bidirectional flows can also be applied to DetNet IP flows.
While the DetNet IP data plane must support bidirectional DetNet While the DetNet IP data plane must support bidirectional DetNet
flows, there are no special bidirectional features with respect to flows, there are no special bidirectional features with respect to
the data plane other than the need for the two directions of a co- the data plane other than the need for the two directions of a
routed bidirectional flow to take the same path. That is to say that co-routed bidirectional flow to take the same path. That is to say,
bidirectional DetNet flows are solely represented at the management bidirectional DetNet flows are solely represented at the management
and control plane levels, without specific support or knowledge plane and control plane levels, without specific support or knowledge
within the DetNet data plane. Fate sharing and associated or co- within the DetNet data plane. Fate-sharing and associated or
routed bidirectional flows, can be managed at the control level. co-routed bidirectional flows can be managed at the control level.
DetNet's use of PREOF may increase the complexity of using co-routing DetNet's use of PREOF may increase the complexity of using co-routing
bidirectional flows, since if PREOF is used, then the replication bidirectional flows, because if PREOF are used, the replication
points in one direction would have to match the elimination points in points in one direction would have to match the elimination points in
the other direction, and vice versa. In such cases the optimal the other direction, and vice versa. In such cases, the optimal
points for these functions in one direction may not match the optimal points for these functions in one direction may not match the optimal
points in the other, due to network and traffic constraints. points in the other, due to network and traffic constraints.
Furthermore, due to the per packet service protection nature, Furthermore, due to the per-packet service protection nature,
bidirectional forwarding per packet may not be ensured. The first bidirectional forwarding may not be ensured. The first packet of
packet of received member flows is selected by the elimination received member flows is selected by the elimination function
function independently of which path it has taken through the independently of which path it has taken through the network.
network.
Control and management mechanisms need to support bidirectional Control and management mechanisms need to support bidirectional
flows, but the specification of such mechanisms are out of scope of flows, but the specification of such mechanisms is out of scope for
this document. Example control plane solutions for MPLS can be found this document. Example control plane solutions for MPLS can be found
in [RFC3473] , [RFC6387] and [RFC7551]. These requirements are in [RFC3473], [RFC6387], and [RFC7551]. These requirements are
included in Section 4.1. included in Section 4.1.
4.3. Packet Replication, Elimination, and Ordering (PREOF) 4.3. Packet Replication, Elimination, and Ordering Functions (PREOF)
The controller plane protocol solution required for managing the The Controller Plane protocol solution required for managing the
PREOF processing is outside the scope of this document. That said, processing of PREOF is outside the scope of this document. That
it should be noted that the ability to determine, for a particular said, it should be noted that the ability to determine, for a
flow, optimal packet replication and elimination points in the DetNet particular flow, optimal packet replication and elimination points in
domain requires explicit support. There may be existing capabilities the DetNet domain requires explicit support. There may be existing
that can be used, or extended, for example GMPLS end-to-end recovery capabilities that can be used or extended -- for example, GMPLS end-
[RFC4872] and GMPLS segment recovery [RFC4873]. to-end recovery [RFC4872] and GMPLS segment recovery [RFC4873].
5. Security Considerations 5. Security Considerations
Security considerations for DetNet are described in detail in Security considerations for DetNet are described in detail in
[I-D.ietf-detnet-security]. General security considerations for [DetNet-Security]. General security considerations for the DetNet
DetNet architecture are described in [RFC8655]. This section architecture are described in [RFC8655]. This section considers
considers architecture-level DetNet security considerations architecture-level DetNet security considerations applicable to all
applicable to all data planes. data planes.
Part of what makes DetNet unique is its ability to provide specific Part of what makes DetNet unique is its ability to provide specific
and reliable quality of service (delivering data flows with extremely and reliable QoS (delivering data flows with extremely low packet
low packet loss rates and bounded end-to-end delivery latency), and loss rates and bounded end-to-end delivery latency), and the
the security-related aspects of protecting that quality of service security-related aspects of protecting that QoS are similarly unique.
are similarly unique.
As for all communications protocols, the primary consideration for As for all communications protocols, the primary consideration for
the data plane is to maintain integrity of data and delivery of the the data plane is to maintain integrity of data and delivery of the
associated DetNet service traversing the DetNet network. Application associated DetNet service traversing the DetNet network. Application
flows can be protected through whatever means is provided by the flows can be protected through whatever means is provided by the
underlying technology. For example, encryption may be used, such as underlying technology. For example, encryption may be used, such as
that provided by IPSec [RFC4301] for IP flows and/or by an underlying that provided by IPsec [RFC4301] for IP flows and/or by an underlying
sub-net using MACSec [IEEE802.1AE-2018] for Ethernet (Layer-2) flows. sub-network using MACsec [IEEE802.1AE-2018] for Ethernet (Layer 2)
flows.
At the management and control level DetNet flows are identified on a At the management and control levels, DetNet flows are identified on
per-flow basis, which may provide controller plane attackers with a per-flow basis, which may provide Controller Plane attackers with
additional information about the data flows (when compared to additional information about the data flows (when compared to
controller planes that do not include per-flow identification). This Controller Planes that do not include per-flow identification). This
is an inherent property of DetNet which has security implications is an inherent property of DetNet that has security implications that
that should be considered when determining if DetNet is a suitable should be considered when determining if DetNet is a suitable
technology for any given use case. technology for any given use case.
To provide uninterrupted availability of the DetNet service, To provide uninterrupted availability of the DetNet service,
provisions can be made against DOS attacks and delay attacks. To provisions can be made against DoS attacks and delay attacks. To
protect against DOS attacks, excess traffic due to malicious or protect against DoS attacks, excess traffic due to malicious or
malfunctioning devices can be prevented or mitigated, for example malfunctioning devices can be prevented or mitigated -- for example,
through the use of existing mechanisms such as policing and shaping through the use of existing mechanisms such as policing and shaping
applied at the input of a DetNet domain. To prevent DetNet packets applied at the input of a DetNet domain. To prevent DetNet packets
from being delayed by an entity external to a DetNet domain, DetNet from being delayed by an entity external to a DetNet domain, DetNet
technology definition can allow for the mitigation of Man-In-The- technology definitions can allow for the mitigation of man-in-the-
Middle attacks, for example through use of authentication and middle attacks -- for example, through the use of authentication and
authorization of devices within the DetNet domain. authorization of devices within the DetNet domain.
In order to prevent or mitigate DetNet attacks on other networks via In order to prevent or mitigate DetNet attacks on other networks via
flow escape, edge devices can for example use existing mechanism such flow escape, edge devices can, for example, use existing mechanisms
as policing and shaping applied at the output of a DetNet domain. such as policing and shaping applied at the output of a DetNet
domain.
6. IANA Considerations 6. IANA Considerations
This document makes no IANA requests. This document has no IANA actions.
7. Acknowledgements
The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson,
David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David
Mozes, Craig Gunther, George Swallow, Yuanlong Jiang and Carlos J.
Bernardos for their various contributions to this work.
8. Contributors
The following people contributed substantially to the content of this
document:
Don Fedyk
Jouni Korhonen
9. References 7. References
9.1. Normative References 7.1. Normative References
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655, "Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019, DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>. <https://www.rfc-editor.org/info/rfc8655>.
9.2. Informative References 7.2. Informative References
[I-D.ietf-detnet-flow-information-model]
Farkas, J., Varga, B., Cummings, R., Jiang, Y., and D.
Fedyk, "DetNet Flow Information Model", draft-ietf-detnet-
flow-information-model-07 (work in progress), March 2020.
[I-D.ietf-detnet-ip] [DetNet-Flow-Info]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., Varga, B., Farkas, J., Cummings, R., Jiang, Y., and D.
and S. Bryant, "DetNet Data Plane: IP", draft-ietf-detnet- Fedyk, "DetNet Flow Information Model", Work in Progress,
ip-05 (work in progress), February 2020. Internet-Draft, draft-ietf-detnet-flow-information-model-
11, 21 October 2020, <https://tools.ietf.org/html/draft-
ietf-detnet-flow-information-model-11>.
[I-D.ietf-detnet-mpls] [DetNet-MPLS]
Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., Varga, B., Ed., Farkas, J., Berger, L., Malis, A., Bryant,
Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", S., and J. Korhonen, "DetNet Data Plane: MPLS", Work in
draft-ietf-detnet-mpls-05 (work in progress), February Progress, Internet-Draft, draft-ietf-detnet-mpls-13, 11
2020. October 2020,
<https://tools.ietf.org/html/draft-ietf-detnet-mpls-13>.
[I-D.ietf-detnet-mpls-over-tsn] [DetNet-MPLS-over-TSN]
Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet Varga, B., Ed., Farkas, J., Malis, A., and S. Bryant,
Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking "DetNet Data Plane: MPLS over IEEE 802.1 Time Sensitive
(TSN)", draft-ietf-detnet-mpls-over-tsn-02 (work in Networking (TSN)", Work in Progress, Internet-Draft,
progress), March 2020. draft-ietf-detnet-mpls-over-tsn-04, 2 November 2020,
<https://tools.ietf.org/html/draft-ietf-detnet-mpls-over-
tsn-04>.
[I-D.ietf-detnet-mpls-over-udp-ip] [DetNet-MPLS-over-UDP-IP]
Varga, B., Farkas, J., Berger, L., Malis, A., and S. Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
Bryant, "DetNet Data Plane: MPLS over UDP/IP", draft-ietf- Bryant, "DetNet Data Plane: MPLS over UDP/IP", Work in
detnet-mpls-over-udp-ip-05 (work in progress), February Progress, Internet-Draft, draft-ietf-detnet-mpls-over-udp-
2020. ip-07, 11 October 2020, <https://tools.ietf.org/html/
draft-ietf-detnet-mpls-over-udp-ip-07>.
[I-D.ietf-detnet-security] [DetNet-Security]
Mizrahi, T. and E. Grossman, "Deterministic Networking Grossman, E., Ed., Mizrahi, T., and A. Hacker,
(DetNet) Security Considerations", draft-ietf-detnet- "Deterministic Networking (DetNet) Security
security-09 (work in progress), March 2020. Considerations", Work in Progress, Internet-Draft, draft-
ietf-detnet-security-12, 2 October 2020,
<https://tools.ietf.org/html/draft-ietf-detnet-security-
12>.
[I-D.ietf-idr-rfc5575bis] [Flow-Spec-Rules]
Loibl, C., Hares, S., Raszuk, R., McPherson, D., and M. Loibl, C., Hares, S., Raszuk, R., McPherson, D., and M.
Bacher, "Dissemination of Flow Specification Rules", Bacher, "Dissemination of Flow Specification Rules", Work
draft-ietf-idr-rfc5575bis-24 (work in progress), April in Progress, Internet-Draft, draft-ietf-idr-rfc5575bis-27,
2020. 15 October 2020, <https://tools.ietf.org/html/draft-ietf-
idr-rfc5575bis-27>.
[I-D.ietf-pce-pcep-extension-for-pce-controller]
Zhao, Q., Li, Z., Negi, M., Peng, S., and C. Zhou, "PCEP
Procedures and Protocol Extensions for Using PCE as a
Central Controller (PCECC) of LSPs", draft-ietf-pce-pcep-
extension-for-pce-controller-04 (work in progress), March
2020.
[I-D.ietf-spring-srv6-network-programming]
Filsfils, C., Camarillo, P., Leddy, J., Voyer, D.,
Matsushima, S., and Z. Li, "SRv6 Network Programming",
draft-ietf-spring-srv6-network-programming-15 (work in
progress), March 2020.
[IEEE802.1AE-2018] [IEEE802.1AE-2018]
IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC IEEE, "IEEE Standard for Local and metropolitan area
Security (MACsec)", 2018, networks-Media Access Control (MAC) Security", IEEE Std
<https://ieeexplore.ieee.org/document/8585421>. 802.1AE-2018, DOI 10.1109/IEEESTD.2018.8585421, December
2018, <https://ieeexplore.ieee.org/document/8585421>.
[IEEE802.1TSNTG] [IEEE802.1TSNTG]
IEEE Standards Association, "IEEE 802.1 Time-Sensitive IEEE, "Time-Sensitive Networking (TSN) Task Group",
Networking Task Group", <http://www.ieee802.org/1/tsn>. <https://1.ieee802.org/tsn/>.
[nwcrg] IRTF, "Coding for efficient NetWork Communications [nwcrg] IRTF, "Coding for efficient NetWork Communications
Research Group (nwcrg)", Research Group (nwcrg)",
<https://datatracker.ietf.org/rg/nwcrg/about>. <https://datatracker.ietf.org/rg/nwcrg/about>.
[PCECC] Li, Z., Peng, S., Negi, M. S., Zhao, Q., and C. Zhou,
"PCEP Procedures and Protocol Extensions for Using PCE as
a Central Controller (PCECC) of LSPs", Work in Progress,
Internet-Draft, draft-ietf-pce-pcep-extension-for-pce-
controller-08, 1 November 2020,
<https://tools.ietf.org/html/draft-ietf-pce-pcep-
extension-for-pce-controller-08>.
[RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
Functional Specification", RFC 2205, DOI 10.17487/RFC2205, Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
September 1997, <https://www.rfc-editor.org/info/rfc2205>. September 1997, <https://www.rfc-editor.org/info/rfc2205>.
[RFC2386] Crawley, E., Nair, R., Rajagopalan, B., and H. Sandick, "A [RFC2386] Crawley, E., Nair, R., Rajagopalan, B., and H. Sandick, "A
Framework for QoS-based Routing in the Internet", Framework for QoS-based Routing in the Internet",
RFC 2386, DOI 10.17487/RFC2386, August 1998, RFC 2386, DOI 10.17487/RFC2386, August 1998,
<https://www.rfc-editor.org/info/rfc2386>. <https://www.rfc-editor.org/info/rfc2386>.
skipping to change at page 24, line 25 skipping to change at line 1118
[RFC3670] Moore, B., Durham, D., Strassner, J., Westerinen, A., and [RFC3670] Moore, B., Durham, D., Strassner, J., Westerinen, A., and
W. Weiss, "Information Model for Describing Network Device W. Weiss, "Information Model for Describing Network Device
QoS Datapath Mechanisms", RFC 3670, DOI 10.17487/RFC3670, QoS Datapath Mechanisms", RFC 3670, DOI 10.17487/RFC3670,
January 2004, <https://www.rfc-editor.org/info/rfc3670>. January 2004, <https://www.rfc-editor.org/info/rfc3670>.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the [RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
December 2005, <https://www.rfc-editor.org/info/rfc4301>. December 2005, <https://www.rfc-editor.org/info/rfc4301>.
[RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, [RFC4872] Lang, J.P., Ed., Rekhter, Y., Ed., and D. Papadimitriou,
"Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for
Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385,
February 2006, <https://www.rfc-editor.org/info/rfc4385>.
[RFC4872] Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou,
Ed., "RSVP-TE Extensions in Support of End-to-End Ed., "RSVP-TE Extensions in Support of End-to-End
Generalized Multi-Protocol Label Switching (GMPLS) Generalized Multi-Protocol Label Switching (GMPLS)
Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007, Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007,
<https://www.rfc-editor.org/info/rfc4872>. <https://www.rfc-editor.org/info/rfc4872>.
[RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel, [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel,
"GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873, "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873,
May 2007, <https://www.rfc-editor.org/info/rfc4873>. May 2007, <https://www.rfc-editor.org/info/rfc4873>.
[RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J.,
skipping to change at page 25, line 39 skipping to change at line 1172
[RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF
Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017,
<https://www.rfc-editor.org/info/rfc8040>. <https://www.rfc-editor.org/info/rfc8040>.
[RFC8227] Cheng, W., Wang, L., Li, H., van Helvoort, H., and J. [RFC8227] Cheng, W., Wang, L., Li, H., van Helvoort, H., and J.
Dong, "MPLS-TP Shared-Ring Protection (MSRP) Mechanism for Dong, "MPLS-TP Shared-Ring Protection (MSRP) Mechanism for
Ring Topology", RFC 8227, DOI 10.17487/RFC8227, August Ring Topology", RFC 8227, DOI 10.17487/RFC8227, August
2017, <https://www.rfc-editor.org/info/rfc8227>. 2017, <https://www.rfc-editor.org/info/rfc8227>.
[RFC8939] Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S.
Bryant, "Deterministic Networking (DetNet) Data Plane:
IP", RFC 8939, DOI 10.17487/RFC8939, November 2020,
<https://www.rfc-editor.org/info/rfc8939>.
[SRv6-Network-Prog]
Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer,
D., Matsushima, S., and Z. Li, "SRv6 Network Programming",
Work in Progress, Internet-Draft, draft-ietf-spring-srv6-
network-programming-26, 26 November 2020,
<https://tools.ietf.org/html/draft-ietf-spring-srv6-
network-programming-26>.
Acknowledgements
The authors wish to thank Pat Thaler, Norman Finn, Loa Andersson,
David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David
Mozes, Craig Gunther, George Swallow, Yuanlong Jiang, and Carlos
J. Bernardos for their various contributions to this work.
Contributors
The following people contributed substantially to the content of this
document:
Don Fedyk
Jouni Korhonen
Authors' Addresses Authors' Addresses
Balazs Varga (editor) Balázs Varga (editor)
Ericsson Ericsson
Budapest
Magyar Tudosok krt. 11. Magyar Tudosok krt. 11.
Budapest 1117 1117
Hungary Hungary
Email: balazs.a.varga@ericsson.com Email: balazs.a.varga@ericsson.com
Janos Farkas
János Farkas
Ericsson Ericsson
Budapest
Magyar Tudosok krt. 11. Magyar Tudosok krt. 11.
Budapest 1117 1117
Hungary Hungary
Email: janos.farkas@ericsson.com Email: janos.farkas@ericsson.com
Lou Berger Lou Berger
LabN Consulting, L.L.C. LabN Consulting, L.L.C.
Email: lberger@labn.net Email: lberger@labn.net
Andrew G. Malis Andrew G. Malis
Malis Consulting Malis Consulting
Email: agmalis@gmail.com Email: agmalis@gmail.com
Stewart Bryant Stewart Bryant
Futurewei Technologies Futurewei Technologies
Email: stewart.bryant@gmail.com Email: sb@stewartbryant.com
 End of changes. 172 change blocks. 
547 lines changed or deleted 574 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/