--- 1/draft-ietf-detnet-use-cases-18.txt 2018-10-08 03:13:15.947861332 -0700 +++ 2/draft-ietf-detnet-use-cases-19.txt 2018-10-08 03:13:16.119865479 -0700 @@ -1,97 +1,90 @@ Internet Engineering Task Force E. Grossman, Ed. Internet-Draft DOLBY -Intended status: Informational September 17, 2018 -Expires: March 21, 2019 +Intended status: Informational October 8, 2018 +Expires: April 11, 2019 Deterministic Networking Use Cases - draft-ietf-detnet-use-cases-18 + draft-ietf-detnet-use-cases-19 Abstract - This draft documents use cases in several diverse industries to - establish multi-hop paths for characterized flows with deterministic - properties. In this context deterministic implies that flows can be - established which provide guaranteed bandwidth and latency which can - be established from either a Layer 2 or Layer 3 (IP) interface, and - which can co-exist on an IP network with best-effort traffic. - - Additional use case properties include optional redundant paths, very - high reliability paths, time synchronization, and clock distribution. - Industries considered include professional audio, electrical - utilities, building automation systems, wireless for industrial, - cellular radio, industrial machine-to-machine, mining, private - blockchain, and network slicing. - - For each case, this document will identify the application, identify - representative solutions used today, and the improvements IETF DetNet - solutions may enable. + This draft presents use cases from diverse industries which have in + common a need for "deterministic flows". "Deterministic" in this + context means that such flows provide guaranteed bandwidth, bounded + latency, and other properties germane to the transport of time- + sensitive data. These use cases differ notably in their network + topologies and specific desired behavior, providing as a group broad + industry context for DetNet. For each use case, this document will + identify the use case, identify representative solutions used today, + and describe potential improvements that DetNet can enable. The Use + Case Common Themes section then extracts and enumerates the set of + common properties implied by these use cases. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on March 21, 2019. + This Internet-Draft will expire on April 11, 2019. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5 - 2. Pro Audio and Video . . . . . . . . . . . . . . . . . . . . . 6 - 2.1. Use Case Description . . . . . . . . . . . . . . . . . . 6 + 2. Pro Audio and Video . . . . . . . . . . . . . . . . . . . . . 7 + 2.1. Use Case Description . . . . . . . . . . . . . . . . . . 7 2.1.1. Uninterrupted Stream Playback . . . . . . . . . . . . 7 - 2.1.2. Synchronized Stream Playback . . . . . . . . . . . . 7 + 2.1.2. Synchronized Stream Playback . . . . . . . . . . . . 8 2.1.3. Sound Reinforcement . . . . . . . . . . . . . . . . . 8 - 2.1.4. Deterministic Time to Establish Streaming . . . . . . 8 - 2.1.5. Secure Transmission . . . . . . . . . . . . . . . . . 8 - 2.1.5.1. Safety . . . . . . . . . . . . . . . . . . . . . 8 + 2.1.4. Secure Transmission . . . . . . . . . . . . . . . . . 9 + 2.1.4.1. Safety . . . . . . . . . . . . . . . . . . . . . 9 2.2. Pro Audio Today . . . . . . . . . . . . . . . . . . . . . 9 2.3. Pro Audio Future . . . . . . . . . . . . . . . . . . . . 9 2.3.1. Layer 3 Interconnecting Layer 2 Islands . . . . . . . 9 - 2.3.2. High Reliability Stream Paths . . . . . . . . . . . . 9 + 2.3.2. High Reliability Stream Paths . . . . . . . . . . . . 10 2.3.3. Integration of Reserved Streams into IT Networks . . 10 2.3.4. Use of Unused Reservations by Best-Effort Traffic . . 10 2.3.5. Traffic Segregation . . . . . . . . . . . . . . . . . 10 2.3.5.1. Packet Forwarding Rules, VLANs and Subnets . . . 11 2.3.5.2. Multicast Addressing (IPv4 and IPv6) . . . . . . 11 2.3.6. Latency Optimization by a Central Controller . . . . 11 - 2.3.7. Reduced Device Cost Due To Reduced Buffer Memory . . 11 + 2.3.7. Reduced Device Cost Due To Reduced Buffer Memory . . 12 2.4. Pro Audio Asks . . . . . . . . . . . . . . . . . . . . . 12 3. Electrical Utilities . . . . . . . . . . . . . . . . . . . . 12 - 3.1. Use Case Description . . . . . . . . . . . . . . . . . . 12 - 3.1.1. Transmission Use Cases . . . . . . . . . . . . . . . 12 + 3.1. Use Case Description . . . . . . . . . . . . . . . . . . 13 + 3.1.1. Transmission Use Cases . . . . . . . . . . . . . . . 13 3.1.1.1. Protection . . . . . . . . . . . . . . . . . . . 13 3.1.1.2. Intra-Substation Process Bus Communications . . . 18 3.1.1.3. Wide Area Monitoring and Control Systems . . . . 19 3.1.1.4. IEC 61850 WAN engineering guidelines requirement classification . . . . . . . . . . . . . . . . . 20 3.1.2. Generation Use Case . . . . . . . . . . . . . . . . . 21 3.1.2.1. Control of the Generated Power . . . . . . . . . 21 3.1.2.2. Control of the Generation Infrastructure . . . . 22 3.1.3. Distribution use case . . . . . . . . . . . . . . . . 27 3.1.3.1. Fault Location Isolation and Service Restoration @@ -181,89 +175,120 @@ 11.1.3. Standardized Data Flow Information Models . . . . . 70 11.1.4. L2 and L3 Integration . . . . . . . . . . . . . . . 70 11.1.5. Consideration for IPv4 . . . . . . . . . . . . . . . 70 11.1.6. Guaranteed End-to-End Delivery . . . . . . . . . . . 70 11.1.7. Replacement for Multiple Proprietary Deterministic Networks . . . . . . . . . . . . . . . . . . . . . . 70 11.1.8. Mix of Deterministic and Best-Effort Traffic . . . . 71 11.1.9. Unused Reserved BW to be Available to Best Effort Traffic . . . . . . . . . . . . . . . . . . . . . . 71 11.1.10. Lower Cost, Multi-Vendor Solutions . . . . . . . . . 71 - 11.2. Scalable Size . . . . . . . . . . . . . . . . . . . . . 71 - 11.3. Scalable Timing Parameters and Accuracy . . . . . . . . 71 - 11.3.1. Bounded Latency . . . . . . . . . . . . . . . . . . 71 + 11.2.1. Scalable Number of Flows . . . . . . . . . . . . . . 71 + 11.3. Scalable Timing Parameters and Accuracy . . . . . . . . 72 + 11.3.1. Bounded Latency . . . . . . . . . . . . . . . . . . 72 11.3.2. Low Latency . . . . . . . . . . . . . . . . . . . . 72 - 11.3.3. Symmetrical Path Delays . . . . . . . . . . . . . . 72 + 11.3.3. Bounded Jitter (Latency Variation) . . . . . . . . . 72 + 11.3.4. Symmetrical Path Delays . . . . . . . . . . . . . . 72 11.4. High Reliability and Availability . . . . . . . . . . . 72 - 11.5. Security . . . . . . . . . . . . . . . . . . . . . . . . 72 + 11.5. Security . . . . . . . . . . . . . . . . . . . . . . . . 73 11.6. Deterministic Flows . . . . . . . . . . . . . . . . . . 73 12. Use Cases Explicitly Out of Scope for DetNet . . . . . . . . 73 12.1. DetNet Scope Limitations . . . . . . . . . . . . . . . . 73 12.2. Internet-based Applications . . . . . . . . . . . . . . 74 12.2.1. Use Case Description . . . . . . . . . . . . . . . . 74 12.2.1.1. Media Content Delivery . . . . . . . . . . . . . 74 12.2.1.2. Online Gaming . . . . . . . . . . . . . . . . . 74 12.2.1.3. Virtual Reality . . . . . . . . . . . . . . . . 74 - 12.2.2. Internet-Based Applications Today . . . . . . . . . 74 - 12.2.3. Internet-Based Applications Future . . . . . . . . . 74 + 12.2.2. Internet-Based Applications Today . . . . . . . . . 75 + 12.2.3. Internet-Based Applications Future . . . . . . . . . 75 12.2.4. Internet-Based Applications Asks . . . . . . . . . . 75 12.3. Pro Audio and Video - Digital Rights Management (DRM) . 75 - 12.4. Pro Audio and Video - Link Aggregation . . . . . . . . . 75 - 13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 76 - 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 77 - 14.1. Pro Audio . . . . . . . . . . . . . . . . . . . . . . . 77 - 14.2. Utility Telecom . . . . . . . . . . . . . . . . . . . . 78 - 14.3. Building Automation Systems . . . . . . . . . . . . . . 78 - 14.4. Wireless for Industrial . . . . . . . . . . . . . . . . 78 - 14.5. Cellular Radio . . . . . . . . . . . . . . . . . . . . . 78 - 14.6. Industrial M2M . . . . . . . . . . . . . . . . . . . . . 79 - 14.7. Internet Applications and CoMP . . . . . . . . . . . . . 79 - 14.8. Electrical Utilities . . . . . . . . . . . . . . . . . . 79 - 14.9. Network Slicing . . . . . . . . . . . . . . . . . . . . 79 - 14.10. Mining . . . . . . . . . . . . . . . . . . . . . . . . . 79 - 14.11. Private Blockchain . . . . . . . . . . . . . . . . . . . 79 - 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 79 - 16. Informative References . . . . . . . . . . . . . . . . . . . 79 - Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 86 + 12.4. Pro Audio and Video - Link Aggregation . . . . . . . . . 76 + 12.5. Pro Audio and Video - Deterministic Time to Establish + Streaming . . . . . . . . . . . . . . . . . . . . . . . 76 + 13. Security Considerations . . . . . . . . . . . . . . . . . . . 76 + 14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 77 + 15. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 78 + 15.1. Pro Audio . . . . . . . . . . . . . . . . . . . . . . . 78 + 15.2. Utility Telecom . . . . . . . . . . . . . . . . . . . . 79 + 15.3. Building Automation Systems . . . . . . . . . . . . . . 79 + 15.4. Wireless for Industrial . . . . . . . . . . . . . . . . 79 + 15.5. Cellular Radio . . . . . . . . . . . . . . . . . . . . . 79 + 15.6. Industrial M2M . . . . . . . . . . . . . . . . . . . . . 79 + 15.7. Internet Applications and CoMP . . . . . . . . . . . . . 80 + 15.8. Network Slicing . . . . . . . . . . . . . . . . . . . . 80 + 15.9. Mining . . . . . . . . . . . . . . . . . . . . . . . . . 80 + 15.10. Private Blockchain . . . . . . . . . . . . . . . . . . . 80 + 16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 80 + 17. Informative References . . . . . . . . . . . . . . . . . . . 80 + Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 87 1. Introduction - This draft presents use cases from diverse industries which have in - common a need for deterministic flows, but which also differ notably - in their network topologies and specific desired behavior. Together, - they provide broad industry context for DetNet and a yardstick - against which proposed DetNet designs can be measured (to what extent - does a proposed design satisfy these various use cases?) + This draft documents use cases in diverse industries which require + deterministic flows over multi-hop paths. DetNet flows can be + established from either a Layer 2 or Layer 3 (IP) interface, and such + flows can co-exist on an IP network with best-effort traffic. DetNet + also provides for highly reliable flows through provision for + redundant paths. - For DetNet, use cases explicitly do not define requirements; The - DetNet WG will consider the use cases, decide which elements are in - scope for DetNet, and the results will be incorporated into future - drafts. Similarly, the DetNet use case draft explicitly does not - suggest any specific design, architecture or protocols, which will be - topics of future drafts. + The DetNet Use Cases explicitly do not suggest any specific design + for DetNet architecture or protocols; these are topics of other + DetNet drafts. - We present for each use case the answers to the following questions: + The DetNet use cases as originally submitted explicitly were not + considered by the DetNet Working Group to be concrete requirements; + The DetNet Working Group and Design Team considered these use cases, + identifying which elements of them could be feasibly implemented + within the charter of DetNet, and as a result certain of the + originally submitted use cases (or elements of them) have been be + moved to the Use Cases Explicitly Out of Scope for DetNet section. + + The DetNet Use Cases document provide context regarding DetNet design + decisions. It also serves a long-lived purpose of helping those + learning (or new to) DetNet to understand the types of applications + that can be supported by DetNet. It also allow those WG contributors + who are users to ensure that their concerns are addressed by the WG; + for them this document both covers their contribution and provides a + long term reference to the problems they expect to be served by the + technology, both in the short term deliverables and as the technology + evolves in the future. + + The DetNet Use Cases document has served as a "yardstick" against + which proposed DetNet designs can be measured, answering the question + "to what extent does a proposed design satisfy these various use + cases?" + + The Use Case industries covered are professional audio, electrical + utilities, building automation systems, wireless for industrial, + cellular radio, industrial machine-to-machine, mining, private + blockchain, and network slicing. For each use case the following + questions are answered: o What is the use case? o How is it addressed today? - o How would you like it to be addressed in the future? + o How should it be addressed in the future? - o What do you want the IETF to deliver? + o What should the IETF deliver to enable this use case? - The level of detail in each use case should be sufficient to express - the relevant elements of the use case, but not more. + The level of detail in each use case is intended to be sufficient to + express the relevant elements of the use case, but not greater than + that. - At the end we consider the use cases collectively, and examine the - most significant goals they have in common. + DetNet does not directly address clock distribution or time + synchronization; these are considered to be part of the overall + design and implementation of a time-sensitive network, using existing + (or future) time-specific protocols (such as [IEEE8021AS] and/or + [RFC5905]). 2. Pro Audio and Video 2.1. Use Case Description The professional audio and video industry ("ProAV") includes: o Music and film content creation o Broadcast @@ -357,50 +382,42 @@ In some cases local performers must perform in synchrony with a remote broadcast. In such cases the latencies of the broadcast stream and the local performer must be adjusted to match each other, with a worst case of one video frame (33ms for NTSC video). In cases where audio phase is a consideration, for example beam- forming using multiple speakers, latency can be in the 10 microsecond range (1 audio sample at 96kHz). -2.1.4. Deterministic Time to Establish Streaming - - Note: The WG has decided that guidelines for deterministic time to - establish stream startup is not within scope of DetNet. If bounded - timing of establishing or re-establish streams is required in a given - use case, it is up to the application/system to achieve this. (The - supporting text from this section has been removed as of draft 12). - -2.1.5. Secure Transmission +2.1.4. Secure Transmission -2.1.5.1. Safety +2.1.4.1. Safety Professional audio systems can include amplifiers that are capable of generating hundreds or thousands of watts of audio power which if used incorrectly can cause hearing damage to those in the vicinity. Apart from the usual care required by the systems operators to prevent such incidents, the network traffic that controls these devices must be secured (as with any sensitive application traffic). 2.2. Pro Audio Today Some proprietary systems have been created which enable deterministic streams at Layer 3 however they are "engineered networks" which require careful configuration to operate, often require that the system be over-provisioned, and it is implied that all devices on the network voluntarily play by the rules of that network. To enable these industries to successfully transition to an interoperable multi-vendor packet-based infrastructure requires effective open - standards, and we believe that establishing relevant IETF standards - is a crucial factor. + standards, and establishing relevant IETF standards is a crucial + factor. 2.3. Pro Audio Future 2.3.1. Layer 3 Interconnecting Layer 2 Islands It would be valuable to enable IP to connect multiple Layer 2 LANs. As an example, ESPN recently constructed a state-of-the-art 194,000 sq ft, $125 million broadcast studio called DC2. The DC2 network is capable of handling 46 Tbps of throughput with 60,000 simultaneous @@ -544,31 +561,31 @@ o Single network for A/V and IT traffic o Standards-based, interoperable, multi-vendor o IT department friendly o Enterprise-wide networks (e.g. size of San Francisco but not the whole Internet (yet...)) 3. Electrical Utilities - 3.1. Use Case Description Many systems that an electrical utility deploys today rely on high availability and deterministic behavior of the underlying networks. - Here we present use cases in Transmission, Generation and - Distribution, including key timing and reliability metrics. We also - discuss security issues and industry trends which affect the - architecture of next generation utility networks + Presented here are use cases in Transmission, Generation and + Distribution, including key timing and reliability metrics. In + addition, security issues and industry trends which affect the + architecture of next generation utility networks are discussed. 3.1.1. Transmission Use Cases + 3.1.1.1. Protection Protection means not only the protection of human operators but also the protection of the electrical equipment and the preservation of the stability and frequency of the grid. If a fault occurs in the transmission or distribution of electricity then severe damage can occur to human operators, electrical equipment and the grid itself, leading to blackouts. Communication links in conjunction with protection relays are used to @@ -958,22 +975,22 @@ | | Mandatory | | Redundancy | Yes | | Packet loss | 1% | +---------------------------------------------------+---------------+ Table 9: FCAG Communication Requirements 3.1.2.2. Control of the Generation Infrastructure The control of the generation infrastructure combines requirements - from industrial automation systems and energy generation systems. In - this section we present the use case of the control of the generation + from industrial automation systems and energy generation systems. + This section considers the use case of the control of the generation infrastructure of a wind turbine. | | | +-----------------+ | | +----+ | | | |WTRM| WGEN | WROT x==|===| | | | | +----+ WCNV| | |WNAC | @@ -1122,32 +1139,32 @@ +--------------+ | XXXXXXX XX | | | | | XX XXXXXXX +----------------+ | | | XXXXX | Wind Park #2 +----+ | | | | +--------------+ Figure 2: Wind Turbine Control via Internet - We expect future use cases which require bounded latency, bounded - jitter and extraordinary low packet loss for inter-domain traffic - flows due to the softwarization and virtualization of core wind farm - equipment (e.g. switches, firewalls and SCADA server components). - These factors will create opportunities for service providers to - install new services and dynamically manage them from remote - locations. For example, to enable fail-over of a local SCADA server, - a SCADA server in another wind farm site (under the administrative - control of the same operator) could be utilized temporarily - (Figure 3). In that case local traffic would be forwarded to the - remote SCADA server and existing intra-domain QoS and timing - parameters would have to be met for inter-domain traffic flows. + Future use cases will require bounded latency, bounded jitter and + extraordinary low packet loss for inter-domain traffic flows due to + the softwarization and virtualization of core wind farm equipment + (e.g. switches, firewalls and SCADA server components). These + factors will create opportunities for service providers to install + new services and dynamically manage them from remote locations. For + example, to enable fail-over of a local SCADA server, a SCADA server + in another wind farm site (under the administrative control of the + same operator) could be utilized temporarily (Figure 3). In that + case local traffic would be forwarded to the remote SCADA server and + existing intra-domain QoS and timing parameters would have to be met + for inter-domain traffic flows. +--------------+ | | | | | Wind Park #1 +----+ | | | XXXXXX | | | X XXXXXXXX +----------------+ +--------------+ | XXXX XXXXX | | +---+ Operator XXX | Remote Control | XXX Administered +----+ Center | @@ -1317,23 +1334,23 @@ IPv6 is seen as a future telecommunications technology for the Smart Grid; the IEC (International Electrotechnical Commission) and different National Committees have mandated a specific adhoc group (AHG8) to define the migration strategy to IPv6 for all the IEC TC57 power automation standards. The AHG8 has recently finalised the work on the migration strategy and the following Technical Report has been issued: IEC TR 62357-200:2015: Guidelines for migration from Internet Protocol version 4 (IPv4) to Internet Protocol version 6 (IPv6). - We expect cloud-based SCADA systems to control and monitor the - critical and non-critical subsystems of generation systems, for - example wind farms. + Cloud-based SCADA systems will control and monitor the critical and + non-critical subsystems of generation systems, for example wind + farms. 3.3.1. Migration to Packet-Switched Network Throughout the world, utilities are increasingly planning for a future based on smart grid applications requiring advanced telecommunications systems. Many of these applications utilize packet connectivity for communicating information and control signals across the utility's Wide Area Network (WAN), made possible by technologies such as multiprotocol label switching (MPLS). The data that traverses the utility WAN includes: @@ -1807,26 +1824,26 @@ so security is definitely a concern, yet security features are not available in the majority of BAS field network deployments . The management network, being an IP-based network, has the protocols available to enable network security, but in practice many BAS systems do not implement even the available security features such as device authentication or encryption for data in transit. 4.3. BAS Future - In the future we expect more fine-grained environmental monitoring - and lower energy consumption, which will require more sensors and + In the future more fine-grained environmental monitoring and lower + energy consumption will emerge which will require more sensors and devices, thus requiring larger and more complex building networks. - We expect building networks to be connected to or converged with - other networks (Enterprise network, Home network, and Internet). + Building networks will be connected to or converged with other + networks (Enterprise network, Home network, and Internet). Therefore better facilities for network management, control, reliability and security are critical in order to improve resident and operator convenience and comfort. For example the ability to monitor and control building devices via the internet would enable (for example) control of room lights or HVAC from a resident's desktop PC or phone application. 4.4. BAS Asks @@ -1878,23 +1895,23 @@ market. For example, a 1% cost reduction in some areas could save $100B 5.1.1. Network Convergence using 6TiSCH Some wireless network technologies support real-time QoS, and are thus useful for these kinds of networks, but others do not. For example WiFi is pervasive but does not provide guaranteed timing or delivery of packets, and thus is not useful in this context. - In this use case we focus on one specific wireless network technology - which does provide the required deterministic QoS, which is "IPv6 - over the TSCH mode of IEEE 802.15.4e" (6TiSCH, where TSCH stands for + This use case focuses on one specific wireless network technology + which provides the required deterministic QoS, which is "IPv6 over + the TSCH mode of IEEE 802.15.4e" (6TiSCH, where TSCH stands for "Time-Slotted Channel Hopping", see [I-D.ietf-6tisch-architecture], [IEEE802154], [IEEE802154e], and [RFC7554]). There are other deterministic wireless busses and networks available today, however they are imcompatible with each other, and incompatible with IP traffic (for example [ISA100], [WirelessHART]). Thus the primary goal of this use case is to apply 6TiSCH as a converged IP- and standards-based wireless network for industrial applications, i.e. to replace multiple proprietary and/or @@ -1950,21 +1967,21 @@ deterministic wireless networks which are incompatible with each other and with IP traffic. 6TiSCH is not yet fully specified, so it cannot be used in today's applications. 5.3. Wireless Industrial Future 5.3.1. Unified Wireless Network and Management - We expect DetNet and 6TiSCH together to enable converged transport of + DetNet and 6TiSCH together can enable converged transport of deterministic and best-effort traffic flows between real-time industrial devices and wide area networks via IP routing. A high level view of a basic such network is shown in Figure 7. ---+-------- ............ ------------ | External Network | | +-----+ +-----+ | NME | | | LLN Border | | | | router +-----+ @@ -1998,24 +2015,24 @@ o | | router | | router | | router +-----+ +-----+ +-----+ o o o o o o o o o o o o o o o o o o o LLN o o o o o o o o o o o o o o o o Figure 8: Extended 6TiSCH Network The backbone router must ensure end-to-end deterministic behavior - between the LLN and the backbone. We would like to see this - accomplished in conformance with the work done in - [I-D.ietf-detnet-architecture] with respect to Layer-3 aspects of - deterministic networks that span multiple Layer-2 domains. + between the LLN and the backbone. This should be accomplished in + conformance with the work done in [I-D.ietf-detnet-architecture] with + respect to Layer-3 aspects of deterministic networks that span + multiple Layer-2 domains. The PCE must compute a deterministic path end-to-end across the TSCH network and IEEE802.1 TSN Ethernet backbone, and DetNet protocols are expected to enable end-to-end deterministic forwarding. +-----+ | IoT | | G/W | +-----+ ^ <---- Elimination @@ -2072,24 +2089,23 @@ solutions, and the forwarding decision is to try the preferred one and use the other in case of Layer-2 transmission failure as detected by ARQ. 5.3.2. Schedule Management by a PCE A common feature of 6TiSCH and DetNet is the action of a PCE to configure paths through the network. Specifically, what is needed is a protocol and data model that the PCE will use to get/set the relevant configuration from/to the devices, as well as perform - operations on the devices. We expect that this protocol will be - developed by DetNet with consideration for its reuse by 6TiSCH. The - remainder of this section provides a bit more context from the 6TiSCH - side. + operations on the devices. This protocol should be developed by + DetNet with consideration for its reuse by 6TiSCH. The remainder of + this section provides a bit more context from the 6TiSCH side. 5.3.2.1. PCE Commands and 6TiSCH CoAP Requests The 6TiSCH device does not expect to place the request for bandwidth between itself and another device in the network. Rather, an operation control system invoked through a human interface specifies the required traffic specification and the end nodes (in terms of latency and reliability). Based on this information, the PCE must compute a path between the end nodes and provision the network with per-flow state that describes the per-hop operation for a given @@ -2109,47 +2125,47 @@ For instance, it is possible that a mapping entity on the backbone transforms a non-CoAP protocol such as PCEP into the RESTful interfaces that the 6TiSCH devices support. This architecture will be refined to comply with DetNet [I-D.ietf-detnet-architecture] when the work is formalized. Related information about 6TiSCH can be found at [I-D.ietf-6tisch-6top-interface] and RPL [RFC6550]. A protocol may be used to update the state in the devices during runtime, for example if it appears that a path through the network has ceased to perform as expected, but in 6TiSCH that flow was not - designed and no protocol was selected. We would like to see DetNet - define the appropriate end-to-end protocols to be used in that case. - The implication is that these state updates take place once the - system is configured and running, i.e. they are not limited to the - initial communication of the configuration of the system. + designed and no protocol was selected. DetNet should define the + appropriate end-to-end protocols to be used in that case. The + implication is that these state updates take place once the system is + configured and running, i.e. they are not limited to the initial + communication of the configuration of the system. A "slotFrame" is the base object that a PCE would manipulate to program a schedule into an LLN node ([I-D.ietf-6tisch-architecture]). - We would like to see the PCE read energy data from devices, and - compute paths that will implement policies on how energy in devices - is consumed, for instance to ensure that the spent energy does not - exceeded the available energy over a period of time. Note: this - statement implies that an extensible protocol for communicating - device info to the PCE and enabling the PCE to act on it will be part - of the DetNet architecture, however for subnets with specific - protocols (e.g. CoAP) a gateway may be required. + The PCE should read energy data from devices and compute paths that + will implement policies on how energy in devices is consumed, for + instance to ensure that the spent energy does not exceeded the + available energy over a period of time. Note: this statement implies + that an extensible protocol for communicating device info to the PCE + and enabling the PCE to act on it will be part of the DetNet + architecture, however for subnets with specific protocols (e.g. + CoAP) a gateway may be required. 6TiSCH devices can discover their neighbors over the radio using a mechanism such as beacons, but even though the neighbor information is available in the 6TiSCH interface data model, 6TiSCH does not describe a protocol to proactively push the neighborhood information - to a PCE. We would like to see DetNet define such a protocol; one - possible design alternative is that it could operate over CoAP, - alternatively it could be converted to/from CoAP by a gateway. We - would like to see such a protocol carry multiple metrics, for example - similar to those used for RPL operations [RFC6551] + to a PCE. DetNet should define such a protocol; one possible design + alternative is that it could operate over CoAP, alternatively it + could be converted to/from CoAP by a gateway. Such a protocol could + carry multiple metrics, for example similar to those used for RPL + operations [RFC6551] 5.3.2.2. 6TiSCH IP Interface "6top" ([I-D.wang-6tisch-6top-sublayer]) is a logical link control sitting between the IP layer and the TSCH MAC layer which provides the link abstraction that is required for IP operations. The 6top data model and management interfaces are further discussed in [I-D.ietf-6tisch-6top-interface] and [I-D.ietf-6tisch-coap]. An IP packet that is sent along a 6TiSCH path uses the Differentiated @@ -2502,21 +2518,21 @@ o All forms of xHaul networks will need some form of DetNet solutions. For example with the advent of 5G some Backhaul traffic will also have DetNet requirements, for example traffic belonging to time-critical 5G applications. o Different splits of the functionality run on the base stations and the on-site units could co-exist on the same Fronthaul and Backhaul network. - We would like to see the following in future Cellular Radio networks: + Future Cellular Radio networks should contain the following: o Unified standards-based transport protocols and standard networking equipment that can make use of underlying deterministic link-layer services o Unified and standards-based network management systems and protocols in all parts of the network (including Fronthaul) New radio access network deployment models and architectures may require time- sensitive networking services with strict requirements @@ -2631,22 +2647,22 @@ networking environment o Aware of underlying deterministic networking services (e.g., on the Ethernet layer) 7. Industrial M2M 7.1. Use Case Description Industrial Automation in general refers to automation of - manufacturing, quality control and material processing. In this - "machine to machine" (M2M) use case we consider machine units in a + manufacturing, quality control and material processing. This + "machine to machine" (M2M) use case considers machine units in a plant floor which periodically exchange data with upstream or downstream machine modules and/or a supervisory controller within a local area network. The actors of M2M communication are Programmable Logic Controllers (PLCs). Communication between PLCs and between PLCs and the supervisory PLC (S-PLC) is achieved via critical control/data streams Figure 11. S (Sensor) @@ -2737,47 +2753,47 @@ Most PLC control loops are rather tolerant of packet loss, however critical control/data streams accept no more than 1 packet loss per consecutive communication cycle (i.e. if a packet gets lost in cycle "n", then the next cycle ("n+1") must be lossless). After two or more consecutive packet losses the network may be considered to be "down" by the Application. As network downtime may impact the whole production system the required network availability is rather high (99,999%). - Based on the above parameters we expect that some form of redundancy - will be required for M2M communications, however any individual - solution depends on several parameters including cycle time, delivery - time, etc. + Based on the above parameters some form of redundancy will be + required for M2M communications, however any individual solution + depends on several parameters including cycle time, delivery time, + etc. 7.2.2. Stream Creation and Destruction In an industrial environment, critical control/data streams are created rather infrequently, on the order of ~10 times per day / week / month. Most of these critical control/data streams get created at machine startup, however flexibility is also needed during runtime, for example when adding or removing a machine. Going forward as - production systems become more flexible, we expect a significant + production systems become more flexible, there will be a significant increase in the rate at which streams are created, changed and destroyed. 7.3. Industrial M2M Future - We would like to see a converged IP-standards-based network with - deterministic properties that can satisfy the timing, security and - reliability constraints described above. Today's proprietary - networks could then be interfaced to such a network via gateways or, - in the case of new installations, devices could be connected directly - to the converged network. + A converged IP-standards-based network with deterministic properties + that can satisfy the timing, security and reliability constraints + described above. Today's proprietary networks could then be + interfaced to such a network via gateways or, in the case of new + installations, devices could be connected directly to the converged + network. - For this use case we expect time synchronization accuracy on the - order of 1us. + For this use case time synchronization accuracy on the order of 1us + is expected. 7.4. Industrial M2M Asks o Converged IP-based network o Deterministic behavior (bounded latency and jitter ) o High availability (presumably through redundancy) (99.999 %) o Low message delivery time (100us - 50ms) @@ -3043,25 +3059,25 @@ techniques used today, which share a physical network among users, do not offer this level of isolation. DetNet can supply point-to-point or point-to-multipoint paths that offer bandwidth and latency guarantees to a user that cannot be affected by other users' data traffic. Thus DetNet is a powerful tool when latency and reliability are required in Network Slicing. 10.2.2. Deterministic Services Within Slices Slices may need to provide services with DetNet-type performance - guarantees, however we note that a system can be implemented to - provide such services in more than one way. For example the slice - itself might be implemented using DetNet, and thus the slice can - provide service guarantees and isolation to its users without any - particular DetNet awareness on the part of the users' applications. + guarantees, however note that a system can be implemented to provide + such services in more than one way. For example the slice itself + might be implemented using DetNet, and thus the slice can provide + service guarantees and isolation to its users without any particular + DetNet awareness on the part of the users' applications. Alternatively, a "non-DetNet-aware" slice may host an application that itself implements DetNet services and thus can enjoy similar service guarantees. 10.3. A Network Slicing Use Case Example - 5G Bearer Network Network Slicing is a core feature of 5G defined in 3GPP, which is currently under development. A network slice in a mobile network is a complete logical network including Radio Access Network (RAN) and Core Network (CN). It provides telecommunication services and @@ -3226,20 +3242,27 @@ 11.2. Scalable Size DetNet networks range in size from very small, e.g. inside a single industrial machine, to very large, for example a Utility Grid network spanning a whole country, and involving many "hops" over various kinds of links for example radio repeaters, microwave linkes, fiber optic links, etc.. However recall that the scope of DetNet is confined to networks that are centrally administered, and explicitly excludes unbounded decentralized networks such as the Internet. +11.2.1. Scalable Number of Flows + + The number of flows in a given network application can potentially be + large, and can potentially grow faster than the number of nodes and + hops. So the network should provide a sufficient (perhaps + configurable) maximum number of flows for any given application. + 11.3. Scalable Timing Parameters and Accuracy 11.3.1. Bounded Latency The DetNet Data Flow Information Model is expected to provide means to configure the network that include parameters for querying network path latency, requesting bounded latency for a given stream, requesting worst case maximum and/or minimum latency for a given path or stream, and so on. It is an expected case that the network may not be able to provide a given requested service level, and if so the @@ -3251,21 +3274,27 @@ Applications may require "extremely low latency" however depending on the application these may mean very different latency values; for example "low latency" across a Utility grid network is on a different time scale than "low latency" in a motor control loop in a small machine. The intent is that the mechanisms for specifying desired latency include wide ranges, and that architecturally there is nothing to prevent arbirtrarily low latencies from being implemented in a given network. -11.3.3. Symmetrical Path Delays +11.3.3. Bounded Jitter (Latency Variation) + + As with the other Latency-related elements noted above, parameters + should be available to determine or request the allowed variation in + latency. + +11.3.4. Symmetrical Path Delays Some applications would like to specify that the transit delay time values be equal for both the transmit and return paths. 11.4. High Reliability and Availability Reliablity is of critical importance to many DetNet applications, in which consequences of failure can be extraordinarily high in terms of cost and even human life. DetNet based systems are expected to be implemented with essentially arbitrarily high availability (for @@ -3373,22 +3402,22 @@ latency is critical to interacting with the virtual world because perceptual delays can cause motion sickness. 12.2.2. Internet-Based Applications Today Internet service today is by definition "best effort", with no guarantees on delivery or bandwidth. 12.2.3. Internet-Based Applications Future - We imagine an Internet from which we will be able to play a video - without glitches and play games without lag. + An Internet from which one can play a video without glitches and play + games without lag. For online gaming, the maximum round-trip delay can be 100ms and stricter for FPS gaming which can be 10-50ms. Transport delay is the dominate part with a 5-20ms budget. For VR, 1-10ms maximum delay is needed and total network budget is 1-5ms if doing remote VR. Flow identification can be used for gaming and VR, i.e. it can recognize a critical flow and provide appropriate latency bounds. @@ -3439,27 +3468,48 @@ with the core goal of achieving the lowest possible latency. For transmitting streams that require more bandwidth than a single link in the target network can support, link aggregation is a technique for combining (aggregating) the bandwidth available on multiple physical links to create a single logical link of the required bandwidth. However, if aggregation is to be used, the network controller (or equivalent) must be able to determine the maximum latency of any path through the aggregate link. -13. Contributors +12.5. Pro Audio and Video - Deterministic Time to Establish Streaming + + The DetNet Working Group has decided that guidelines for establishing + a deterministic time to establish stream startup are not within scope + of DetNet. If bounded timing of establishing or re-establish streams + is required in a given use case, it is up to the application/system + to achieve this. + +13. Security Considerations + + This document covers a number of representative applications and + network scenarios that are expected to make use of DetNet + technologies. Each of the potential DetNet uses cases will have + security considerations from both the use-specific and DetNet + technology perspectives. While some use-specific security + considerations are discussed above, a more comprehensive discussion + of such considerations is captured in DetNet Security Considerations + [I-D.ietf-detnet-security]. Readers are encouraged to review this + document to gain a more complete understanding of DetNet related + security considerations. + +14. Contributors RFC7322 limits the number of authors listed on the front page of a draft to a maximum of 5, far fewer than the 20 individuals below who made important contributions to this draft. The editor wishes to thank and acknowledge each of the following authors for contributing - text to this draft. See also Section 14. + text to this draft. See also Section 15. Craig Gunther (Harman International) 10653 South River Front Parkway, South Jordan,UT 84095 phone +1 801 568-7675, email craig.gunther@harman.com Pascal Thubert (Cisco Systems, Inc) Building D, 45 Allee des Ormes - BP1200, MOUGINS Sophia Antipolis 06254 FRANCE phone +33 497 23 26 34, email pthubert@cisco.com @@ -3520,23 +3570,23 @@ Xuesong Geng (Huawei Technologies) email gengxuesong@huawei.com Diego Dujovne (Universidad Diego Portales) email diego.dujovne@mail.udp.cl Maik Seewald (Cisco Systems) email maseewal@cisco.com -14. Acknowledgments +15. Acknowledgments -14.1. Pro Audio +15.1. Pro Audio This section was derived from draft-gunther-detnet-proaudio-req-01. The editors would like to acknowledge the help of the following individuals and the companies they represent: Jeff Koftinoff, Meyer Sound Jouni Korhonen, Associate Technical Director, Broadcom @@ -3534,99 +3584,96 @@ This section was derived from draft-gunther-detnet-proaudio-req-01. The editors would like to acknowledge the help of the following individuals and the companies they represent: Jeff Koftinoff, Meyer Sound Jouni Korhonen, Associate Technical Director, Broadcom Pascal Thubert, CTAO, Cisco - Kieran Tyrrell, Sienda New Media Technologies GmbH -14.2. Utility Telecom +15.2. Utility Telecom This section was derived from draft-wetterwald-detnet-utilities-reqs- 02. Faramarz Maghsoodlou, Ph. D. IoT Connected Industries and Energy Practice Cisco Pascal Thubert, CTAO Cisco -14.3. Building Automation Systems + The wind power generation use case has been extracted from the study + of Wind Farms conducted within the 5GPPP Virtuwind Project. The + project is funded by the European Union's Horizon 2020 research and + innovation programme under grant agreement No 671648 (VirtuWind). + +15.3. Building Automation Systems This section was derived from draft-bas-usecase-detnet-00. -14.4. Wireless for Industrial +15.4. Wireless for Industrial This section was derived from draft-thubert-6tisch-4detnet-01. This specification derives from the 6TiSCH architecture, which is the result of multiple interactions, in particular during the 6TiSCH (bi)Weekly Interim call, relayed through the 6TiSCH mailing list at the IETF. The authors wish to thank: Kris Pister, Thomas Watteyne, Xavier Vilajosana, Qin Wang, Tom Phinney, Robert Assimiti, Michael Richardson, Zhuo Chen, Malisa Vucinic, Alfredo Grieco, Martin Turon, Dominique Barthel, Elvis Vogli, Guillaume Gaillard, Herman Storey, Maria Rita Palattella, Nicola Accettura, Patrick Wetterwald, Pouria Zand, Raghuram Sudhaakar, and Shitanshu Shah for their participation and various contributions. -14.5. Cellular Radio +15.5. Cellular Radio This section was derived from draft-korhonen-detnet-telreq-00. -14.6. Industrial M2M +15.6. Industrial M2M The authors would like to thank Feng Chen and Marcel Kiessling for their comments and suggestions. -14.7. Internet Applications and CoMP +15.7. Internet Applications and CoMP This section was derived from draft-zha-detnet-use-case-00 by Yiyong Zha. This document has benefited from reviews, suggestions, comments and proposed text provided by the following members, listed in alphabetical order: Jing Huang, Junru Lin, Lehong Niu and Oilver Huang. -14.8. Electrical Utilities - - The wind power generation use case has been extracted from the study - of Wind Farms conducted within the 5GPPP Virtuwind Project. The - project is funded by the European Union's Horizon 2020 research and - innovation programme under grant agreement No 671648 (VirtuWind). - -14.9. Network Slicing +15.8. Network Slicing This section was written by Xuesong Geng, who would like to acknowledge Norm Finn and Mach Chen for their useful comments. -14.10. Mining +15.9. Mining This section was written by Diego Dujovne in conjunction with Xavier Vilasojana. -14.11. Private Blockchain +15.10. Private Blockchain This section was written by Daniel Huang. -15. IANA Considerations +16. IANA Considerations This memo includes no requests from IANA. -16. Informative References +17. Informative References [Ahm14] Ahmed, M. and R. Kim, "Communication network architectures for smart-wind power farms.", Energies, p. 3900-3921. , June 2014. [bacnetip] ASHRAE, "Annex J to ANSI/ASHRAE 135-1995 - BACnet/IP", January 1999. [CoMP] NGMN Alliance, "RAN EVOLUTION PROJECT COMP EVALUATION AND @@ -3677,26 +3724,32 @@ in progress), April 2018. [I-D.ietf-6tisch-coap] Sudhaakar, R. and P. Zand, "6TiSCH Resource Management and Interaction using CoAP", draft-ietf-6tisch-coap-03 (work in progress), March 2015. [I-D.ietf-detnet-architecture] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", draft-ietf- - detnet-architecture-07 (work in progress), August 2018. + detnet-architecture-08 (work in progress), September 2018. [I-D.ietf-detnet-problem-statement] Finn, N. and P. Thubert, "Deterministic Networking Problem - Statement", draft-ietf-detnet-problem-statement-06 (work - in progress), July 2018. + Statement", draft-ietf-detnet-problem-statement-07 (work + in progress), October 2018. + + [I-D.ietf-detnet-security] + Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell, + J., Austad, H., Stanton, K., and N. Finn, "Deterministic + Networking (DetNet) Security Considerations", draft-ietf- + detnet-security-02 (work in progress), April 2018. [I-D.ietf-tictoc-1588overmpls] Davari, S., Oren, A., Bhatia, M., Roberts, P., and L. Montini, "Transporting Timing messages over MPLS Networks", draft-ietf-tictoc-1588overmpls-07 (work in progress), October 2015. [I-D.kh-spring-ip-ran-use-case] Khasnabish, B., hu, f., and L. Contreras, "Segment Routing in IP RAN use case", draft-kh-spring-ip-ran-use-case-02 @@ -3854,20 +3907,25 @@ P. Pate, "Structure-Aware Time Division Multiplexed (TDM) Circuit Emulation Service over Packet Switched Network (CESoPSN)", RFC 5086, DOI 10.17487/RFC5086, December 2007, . [RFC5087] Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi, "Time Division Multiplexing over IP (TDMoIP)", RFC 5087, DOI 10.17487/RFC5087, December 2007, . + [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, + "Network Time Protocol Version 4: Protocol and Algorithms + Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, + . + [RFC6550] Winter, T., Ed., Thubert, P., Ed., Brandt, A., Hui, J., Kelsey, R., Levis, P., Pister, K., Struik, R., Vasseur, JP., and R. Alexander, "RPL: IPv6 Routing Protocol for Low-Power and Lossy Networks", RFC 6550, DOI 10.17487/RFC6550, March 2012, . [RFC6551] Vasseur, JP., Ed., Kim, M., Ed., Pister, K., Dejean, N., and D. Barthel, "Routing Metrics Used for Path Calculation in Low-Power and Lossy Networks", RFC 6551,