--- 1/draft-ietf-rtcweb-jsep-18.txt 2017-03-10 12:13:45.113386037 -0800 +++ 2/draft-ietf-rtcweb-jsep-19.txt 2017-03-10 12:13:45.321390917 -0800 @@ -1,21 +1,21 @@ Network Working Group J. Uberti Internet-Draft Google Intended status: Standards Track C. Jennings -Expires: July 20, 2017 Cisco +Expires: September 11, 2017 Cisco E. Rescorla, Ed. Mozilla - January 16, 2017 + March 10, 2017 Javascript Session Establishment Protocol - draft-ietf-rtcweb-jsep-18 + draft-ietf-rtcweb-jsep-19 Abstract This document describes the mechanisms for allowing a Javascript application to control the signaling plane of a multimedia session via the interface specified in the W3C RTCPeerConnection API, and discusses how this relates to existing signaling protocols. Status of This Memo @@ -25,21 +25,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on July 20, 2017. + This Internet-Draft will expire on September 11, 2017. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -51,106 +51,106 @@ Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. General Design of JSEP . . . . . . . . . . . . . . . . . 4 1.2. Other Approaches Considered . . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Semantics and Syntax . . . . . . . . . . . . . . . . . . . . 6 3.1. Signaling Model . . . . . . . . . . . . . . . . . . . . . 6 3.2. Session Descriptions and State Machine . . . . . . . . . 7 - 3.3. Session Description Format . . . . . . . . . . . . . . . 10 - 3.4. Session Description Control . . . . . . . . . . . . . . . 10 - 3.4.1. RtpTransceivers . . . . . . . . . . . . . . . . . . . 10 - 3.4.2. RtpSenders . . . . . . . . . . . . . . . . . . . . . 11 - 3.4.3. RtpReceivers . . . . . . . . . . . . . . . . . . . . 11 - 3.5. ICE . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 - 3.5.1. ICE Gathering Overview . . . . . . . . . . . . . . . 11 - 3.5.2. ICE Candidate Trickling . . . . . . . . . . . . . . . 12 - 3.5.2.1. ICE Candidate Format . . . . . . . . . . . . . . 12 - 3.5.3. ICE Candidate Policy . . . . . . . . . . . . . . . . 13 - 3.5.4. ICE Candidate Pool . . . . . . . . . . . . . . . . . 14 - 3.6. Video Size Negotiation . . . . . . . . . . . . . . . . . 15 - 3.6.1. Creating an imageattr Attribute . . . . . . . . . . . 15 - 3.6.2. Interpreting an imageattr Attribute . . . . . . . . . 16 - 3.7. Simulcast . . . . . . . . . . . . . . . . . . . . . . . . 17 - 3.8. Interactions With Forking . . . . . . . . . . . . . . . . 18 - 3.8.1. Sequential Forking . . . . . . . . . . . . . . . . . 19 - 3.8.2. Parallel Forking . . . . . . . . . . . . . . . . . . 19 - 4. Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 20 - 4.1. PeerConnection . . . . . . . . . . . . . . . . . . . . . 20 - 4.1.1. Constructor . . . . . . . . . . . . . . . . . . . . . 20 - 4.1.2. addTrack . . . . . . . . . . . . . . . . . . . . . . 22 - 4.1.3. removeTrack . . . . . . . . . . . . . . . . . . . . . 23 - 4.1.4. addTransceiver . . . . . . . . . . . . . . . . . . . 23 - 4.1.5. createDataChannel . . . . . . . . . . . . . . . . . . 23 + 3.3. Session Description Format . . . . . . . . . . . . . . . 11 + 3.4. Session Description Control . . . . . . . . . . . . . . . 11 + 3.4.1. RtpTransceivers . . . . . . . . . . . . . . . . . . . 11 + 3.4.2. RtpSenders . . . . . . . . . . . . . . . . . . . . . 12 + 3.4.3. RtpReceivers . . . . . . . . . . . . . . . . . . . . 12 + 3.5. ICE . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 + 3.5.1. ICE Gathering Overview . . . . . . . . . . . . . . . 12 + 3.5.2. ICE Candidate Trickling . . . . . . . . . . . . . . . 13 + 3.5.2.1. ICE Candidate Format . . . . . . . . . . . . . . 13 + 3.5.3. ICE Candidate Policy . . . . . . . . . . . . . . . . 14 + 3.5.4. ICE Candidate Pool . . . . . . . . . . . . . . . . . 15 + 3.6. Video Size Negotiation . . . . . . . . . . . . . . . . . 16 + 3.6.1. Creating an imageattr Attribute . . . . . . . . . . . 16 + 3.6.2. Interpreting an imageattr Attribute . . . . . . . . . 17 + 3.7. Simulcast . . . . . . . . . . . . . . . . . . . . . . . . 18 + 3.8. Interactions With Forking . . . . . . . . . . . . . . . . 19 + 3.8.1. Sequential Forking . . . . . . . . . . . . . . . . . 20 + 3.8.2. Parallel Forking . . . . . . . . . . . . . . . . . . 20 + 4. Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 21 + 4.1. PeerConnection . . . . . . . . . . . . . . . . . . . . . 21 + 4.1.1. Constructor . . . . . . . . . . . . . . . . . . . . . 21 + 4.1.2. addTrack . . . . . . . . . . . . . . . . . . . . . . 23 + 4.1.3. removeTrack . . . . . . . . . . . . . . . . . . . . . 24 + 4.1.4. addTransceiver . . . . . . . . . . . . . . . . . . . 24 + 4.1.5. createDataChannel . . . . . . . . . . . . . . . . . . 24 4.1.6. createOffer . . . . . . . . . . . . . . . . . . . . . 24 4.1.7. createAnswer . . . . . . . . . . . . . . . . . . . . 25 - 4.1.8. SessionDescriptionType . . . . . . . . . . . . . . . 25 - 4.1.8.1. Use of Provisional Answers . . . . . . . . . . . 26 - 4.1.8.2. Rollback . . . . . . . . . . . . . . . . . . . . 27 - 4.1.9. setLocalDescription . . . . . . . . . . . . . . . . . 28 - 4.1.10. setRemoteDescription . . . . . . . . . . . . . . . . 28 - 4.1.11. currentLocalDescription . . . . . . . . . . . . . . . 29 - 4.1.12. pendingLocalDescription . . . . . . . . . . . . . . . 29 - 4.1.13. currentRemoteDescription . . . . . . . . . . . . . . 29 - 4.1.14. pendingRemoteDescription . . . . . . . . . . . . . . 29 - 4.1.15. canTrickleIceCandidates . . . . . . . . . . . . . . . 30 - 4.1.16. setConfiguration . . . . . . . . . . . . . . . . . . 30 - 4.1.17. addIceCandidate . . . . . . . . . . . . . . . . . . . 31 - 4.2. RtpTransceiver . . . . . . . . . . . . . . . . . . . . . 32 - 4.2.1. stop . . . . . . . . . . . . . . . . . . . . . . . . 32 - 4.2.2. stopped . . . . . . . . . . . . . . . . . . . . . . . 32 - 4.2.3. setDirection . . . . . . . . . . . . . . . . . . . . 32 - 4.2.4. direction . . . . . . . . . . . . . . . . . . . . . . 32 - 4.2.5. currentDirection . . . . . . . . . . . . . . . . . . 33 - 4.2.6. setCodecPreferences . . . . . . . . . . . . . . . . . 33 - 5. SDP Interaction Procedures . . . . . . . . . . . . . . . . . 33 - 5.1. Requirements Overview . . . . . . . . . . . . . . . . . . 34 - 5.1.1. Implementation Requirements . . . . . . . . . . . . . 34 - 5.1.2. Usage Requirements . . . . . . . . . . . . . . . . . 35 - 5.1.3. Profile Names and Interoperability . . . . . . . . . 36 - 5.2. Constructing an Offer . . . . . . . . . . . . . . . . . . 37 - 5.2.1. Initial Offers . . . . . . . . . . . . . . . . . . . 37 - 5.2.2. Subsequent Offers . . . . . . . . . . . . . . . . . . 42 - 5.2.3. Options Handling . . . . . . . . . . . . . . . . . . 46 - 5.2.3.1. IceRestart . . . . . . . . . . . . . . . . . . . 46 - 5.2.3.2. VoiceActivityDetection . . . . . . . . . . . . . 46 - 5.3. Generating an Answer . . . . . . . . . . . . . . . . . . 47 - 5.3.1. Initial Answers . . . . . . . . . . . . . . . . . . . 47 - 5.3.2. Subsequent Answers . . . . . . . . . . . . . . . . . 51 - 5.3.3. Options Handling . . . . . . . . . . . . . . . . . . 53 - 5.3.3.1. VoiceActivityDetection . . . . . . . . . . . . . 53 - 5.4. Modifying an Offer or Answer . . . . . . . . . . . . . . 53 - 5.5. Processing a Local Description . . . . . . . . . . . . . 54 - 5.6. Processing a Remote Description . . . . . . . . . . . . . 54 - 5.7. Parsing a Session Description . . . . . . . . . . . . . . 55 - 5.7.1. Session-Level Parsing . . . . . . . . . . . . . . . . 55 - 5.7.2. Media Section Parsing . . . . . . . . . . . . . . . . 57 - 5.7.3. Semantics Verification . . . . . . . . . . . . . . . 59 - 5.8. Applying a Local Description . . . . . . . . . . . . . . 60 - 5.9. Applying a Remote Description . . . . . . . . . . . . . . 62 - 5.10. Applying an Answer . . . . . . . . . . . . . . . . . . . 65 - 6. Processing RTP/RTCP . . . . . . . . . . . . . . . . . . . . . 68 - 7. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 68 - 7.1. Simple Example . . . . . . . . . . . . . . . . . . . . . 68 - 7.2. Normal Examples . . . . . . . . . . . . . . . . . . . . . 72 - 8. Security Considerations . . . . . . . . . . . . . . . . . . . 81 - 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 81 - 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 81 - 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 82 - 11.1. Normative References . . . . . . . . . . . . . . . . . . 82 - 11.2. Informative References . . . . . . . . . . . . . . . . . 85 - Appendix A. Appendix A . . . . . . . . . . . . . . . . . . . . . 87 - Appendix B. Appendix B . . . . . . . . . . . . . . . . . . . . . 88 - Appendix C. Change log . . . . . . . . . . . . . . . . . . . . . 91 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 99 + 4.1.8. SessionDescriptionType . . . . . . . . . . . . . . . 26 + 4.1.8.1. Use of Provisional Answers . . . . . . . . . . . 27 + 4.1.8.2. Rollback . . . . . . . . . . . . . . . . . . . . 28 + 4.1.9. setLocalDescription . . . . . . . . . . . . . . . . . 29 + 4.1.10. setRemoteDescription . . . . . . . . . . . . . . . . 29 + 4.1.11. currentLocalDescription . . . . . . . . . . . . . . . 30 + 4.1.12. pendingLocalDescription . . . . . . . . . . . . . . . 30 + 4.1.13. currentRemoteDescription . . . . . . . . . . . . . . 30 + 4.1.14. pendingRemoteDescription . . . . . . . . . . . . . . 30 + 4.1.15. canTrickleIceCandidates . . . . . . . . . . . . . . . 31 + 4.1.16. setConfiguration . . . . . . . . . . . . . . . . . . 31 + 4.1.17. addIceCandidate . . . . . . . . . . . . . . . . . . . 32 + 4.2. RtpTransceiver . . . . . . . . . . . . . . . . . . . . . 33 + 4.2.1. stop . . . . . . . . . . . . . . . . . . . . . . . . 33 + 4.2.2. stopped . . . . . . . . . . . . . . . . . . . . . . . 33 + 4.2.3. setDirection . . . . . . . . . . . . . . . . . . . . 33 + 4.2.4. direction . . . . . . . . . . . . . . . . . . . . . . 33 + 4.2.5. currentDirection . . . . . . . . . . . . . . . . . . 34 + 4.2.6. setCodecPreferences . . . . . . . . . . . . . . . . . 34 + 5. SDP Interaction Procedures . . . . . . . . . . . . . . . . . 34 + 5.1. Requirements Overview . . . . . . . . . . . . . . . . . . 35 + 5.1.1. Usage Requirements . . . . . . . . . . . . . . . . . 35 + 5.1.2. Profile Names and Interoperability . . . . . . . . . 35 + 5.2. Constructing an Offer . . . . . . . . . . . . . . . . . . 36 + 5.2.1. Initial Offers . . . . . . . . . . . . . . . . . . . 36 + 5.2.2. Subsequent Offers . . . . . . . . . . . . . . . . . . 43 + 5.2.3. Options Handling . . . . . . . . . . . . . . . . . . 47 + 5.2.3.1. IceRestart . . . . . . . . . . . . . . . . . . . 47 + 5.2.3.2. VoiceActivityDetection . . . . . . . . . . . . . 47 + 5.3. Generating an Answer . . . . . . . . . . . . . . . . . . 48 + 5.3.1. Initial Answers . . . . . . . . . . . . . . . . . . . 48 + 5.3.2. Subsequent Answers . . . . . . . . . . . . . . . . . 54 + 5.3.3. Options Handling . . . . . . . . . . . . . . . . . . 55 + 5.3.3.1. VoiceActivityDetection . . . . . . . . . . . . . 56 + 5.4. Modifying an Offer or Answer . . . . . . . . . . . . . . 56 + 5.5. Processing a Local Description . . . . . . . . . . . . . 57 + 5.6. Processing a Remote Description . . . . . . . . . . . . . 57 + 5.7. Parsing a Session Description . . . . . . . . . . . . . . 58 + 5.7.1. Session-Level Parsing . . . . . . . . . . . . . . . . 58 + 5.7.2. Media Section Parsing . . . . . . . . . . . . . . . . 60 + 5.7.3. Semantics Verification . . . . . . . . . . . . . . . 62 + 5.8. Applying a Local Description . . . . . . . . . . . . . . 64 + 5.9. Applying a Remote Description . . . . . . . . . . . . . . 65 + 5.10. Applying an Answer . . . . . . . . . . . . . . . . . . . 69 + 6. Processing RTP/RTCP . . . . . . . . . . . . . . . . . . . . . 71 + 7. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 71 + 7.1. Simple Example . . . . . . . . . . . . . . . . . . . . . 72 + 7.2. Detailed Example . . . . . . . . . . . . . . . . . . . . 77 + 7.3. Early Transport Warmup Example . . . . . . . . . . . . . 86 + 8. Security Considerations . . . . . . . . . . . . . . . . . . . 94 + 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 95 + 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 95 + 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 95 + 11.1. Normative References . . . . . . . . . . . . . . . . . . 95 + 11.2. Informative References . . . . . . . . . . . . . . . . . 99 + Appendix A. Appendix A . . . . . . . . . . . . . . . . . . . . . 101 + Appendix B. Appendix B . . . . . . . . . . . . . . . . . . . . . 102 + Appendix C. Change log . . . . . . . . . . . . . . . . . . . . . 107 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 116 1. Introduction This document describes how the W3C WEBRTC RTCPeerConnection interface [W3C.WD-webrtc-20140617] is used to control the setup, management and teardown of a multimedia session. 1.1. General Design of JSEP The thinking behind WebRTC call setup has been to fully specify and @@ -159,61 +159,66 @@ applications may prefer to use different protocols, such as the existing SIP or Jingle call signaling protocols, or something custom to the particular application, perhaps for a novel use case. In this approach, the key information that needs to be exchanged is the multimedia session description, which specifies the necessary transport and media configuration information necessary to establish the media plane. With these considerations in mind, this document describes the Javascript Session Establishment Protocol (JSEP) that allows for full - control of the signaling state machine from Javascript. JSEP removes - the browser almost entirely from the core signaling flow, which is - instead handled by the Javascript making use of two interfaces: (1) - passing in local and remote session descriptions and (2) interacting - with the ICE state machine. + control of the signaling state machine from Javascript. As described + above, JSEP assumes a model in which a Javascript application + executes inside a runtime containing WebRTC APIs (the "JSEP + implementation"). The JSEP implementation is almost entirely + divorced from the core signaling flow, which is instead handled by + the Javascript making use of two interfaces: (1) passing in local and + remote session descriptions and (2) interacting with the ICE state + machine. The combination of the JSEP implementation and the + Javascript application is referred to throughout this document as a + "JSEP endpoint". In this document, the use of JSEP is described as if it always occurs - between two browsers. Note though in many cases it will actually be - between a browser and some kind of server, such as a gateway or MCU. - This distinction is invisible to the browser; it just follows the - instructions it is given via the API. + between two JSEP endpoints. Note though in many cases it will + actually be between a JSEP endpoint and some kind of server, such as + a gateway or MCU. This distinction is invisible to the JSEP + endpoint; it just follows the instructions it is given via the API. JSEP's handling of session descriptions is simple and straightforward. Whenever an offer/answer exchange is needed, the initiating side creates an offer by calling a createOffer() API. The application then uses that offer to set up its local config via the setLocalDescription() API. The offer is finally sent off to the remote side over its preferred signaling mechanism (e.g., WebSockets); upon receipt of that offer, the remote party installs it using the setRemoteDescription() API. To complete the offer/answer exchange, the remote party uses the createAnswer() API to generate an appropriate answer, applies it using the setLocalDescription() API, and sends the answer back to the initiator over the signaling channel. When the initiator gets that answer, it installs it using the setRemoteDescription() API, and initial setup is complete. This process can be repeated for additional offer/answer exchanges. Regarding ICE [RFC5245], JSEP decouples the ICE state machine from the overall signaling state machine, as the ICE state machine must - remain in the browser, because only the browser has the necessary - knowledge of candidates and other transport info. Performing this - separation also provides additional flexibility; in protocols that - decouple session descriptions from transport, such as Jingle, the - session description can be sent immediately and the transport - information can be sent when available. In protocols that don't, - such as SIP, the information can be used in the aggregated form. - Sending transport information separately can allow for faster ICE and - DTLS startup, since ICE checks can start as soon as any transport - information is available rather than waiting for all of it. + remain in the JSEP implementation, because only the implementation + has the necessary knowledge of candidates and other transport info. + Performing this separation also provides additional flexibility; in + protocols that decouple session descriptions from transport, such as + Jingle, the session description can be sent immediately and the + transport information can be sent when available. In protocols that + don't, such as SIP, the information can be used in the aggregated + form. Sending transport information separately can allow for faster + ICE and DTLS startup, since ICE checks can start as soon as any + transport information is available rather than waiting for all of it. Through its abstraction of signaling, the JSEP approach does require the application to be aware of the signaling process. While the application does not need to understand the contents of session descriptions to set up a call, the application must call the right APIs at the right times, convert the session descriptions and ICE information into the defined messages of its chosen signaling protocol, and perform the reverse conversion on the messages it receives from the other side. @@ -225,116 +230,115 @@ the JSEP API into an API suitable for a SIP or XMPP. Thus, JSEP provides greater control for the experienced developer without forcing any additional complexity on the novice developer. 1.2. Other Approaches Considered One approach that was considered instead of JSEP was to include a lightweight signaling protocol. Instead of providing session descriptions to the API, the API would produce and consume messages from this protocol. While providing a more high-level API, this put - more control of signaling within the browser, forcing the browser to - have to understand and handle concepts like signaling glare. In + more control of signaling within the JSEP implementation, forcing it + to have to understand and handle concepts like signaling glare. In addition, it prevented the application from driving the state machine to a desired state, as is needed in the page reload case. A second approach that was considered but not chosen was to decouple the management of the media control objects from session descriptions, instead offering APIs that would control each component directly. This was rejected based on a feeling that requiring exposure of this level of complexity to the application programmer would not be beneficial; it would result in an API where even a simple example would require a significant amount of code to orchestrate all the needed interactions, as well as creating a large API surface that needed to be agreed upon and documented. In addition, these API points could be called in any order, resulting in a more complex set of interactions with the media subsystem than the JSEP approach, which specifies how session descriptions are to be evaluated and applied. One variation on JSEP that was considered was to keep the basic session description-oriented API, but to move the mechanism for - generating offers and answers out of the browser. Instead of - providing createOffer/createAnswer methods within the browser, this - approach would instead expose a getCapabilities API which would - provide the application with the information it needed in order to - generate its own session descriptions. This increases the amount of - work that the application needs to do; it needs to know how to - generate session descriptions from capabilities, and especially how - to generate the correct answer from an arbitrary offer and the - supported capabilities. While this could certainly be addressed by - using a library like the one mentioned above, it basically forces the - use of said library even for a simple example. Providing - createOffer/createAnswer avoids this problem, but still allows - applications to generate their own offers/answers (to a large extent) - if they choose, using the description generated by createOffer as an - indication of the browser's capabilities. + generating offers and answers out of the JSEP implementation. + Instead of providing createOffer/createAnswer methods within the + implementation, this approach would instead expose a getCapabilities + API which would provide the application with the information it + needed in order to generate its own session descriptions. This + increases the amount of work that the application needs to do; it + needs to know how to generate session descriptions from capabilities, + and especially how to generate the correct answer from an arbitrary + offer and the supported capabilities. While this could certainly be + addressed by using a library like the one mentioned above, it + basically forces the use of said library even for a simple example. + Providing createOffer/createAnswer avoids this problem. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 3. Semantics and Syntax 3.1. Signaling Model JSEP does not specify a particular signaling model or state machine, other than the generic need to exchange session descriptions in the fashion described by [RFC3264](offer/answer) in order for both sides of the session to know how to conduct the session. JSEP provides mechanisms to create offers and answers, as well as to apply them to - a session. However, the browser is totally decoupled from the actual - mechanism by which these offers and answers are communicated to the - remote side, including addressing, retransmission, forking, and glare - handling. These issues are left entirely up to the application; the - application has complete control over which offers and answers get - handed to the browser, and when. + a session. However, the JSEP implementation is totally decoupled + from the actual mechanism by which these offers and answers are + communicated to the remote side, including addressing, + retransmission, forking, and glare handling. These issues are left + entirely up to the application; the application has complete control + over which offers and answers get handed to the implementation, and + when. +-----------+ +-----------+ | Web App |<--- App-Specific Signaling -->| Web App | +-----------+ +-----------+ ^ ^ | SDP | SDP V V +-----------+ +-----------+ - | Browser |<----------- Media ------------>| Browser | + | JSEP |<----------- Media ------------>| JSEP | + | Impl. | | Impl. | +-----------+ +-----------+ Figure 1: JSEP Signaling Model 3.2. Session Descriptions and State Machine In order to establish the media plane, the user agent needs specific parameters to indicate what to transmit to the remote side, as well as how to handle the media that is received. These parameters are determined by the exchange of session descriptions in offers and answers, and there are certain details to this process that must be handled in the JSEP APIs. Whether a session description applies to the local side or the remote side affects the meaning of that description. For example, the list of codecs sent to a remote party indicates what the local side is willing to receive, which, when intersected with the set of codecs the remote side supports, specifies what the remote side should send. - However, not all parameters follow this rule; for example, the DTLS- - SRTP parameters [RFC5763] sent to a remote party indicate what - certificate the local side will use in DTLS setup, and thereby what - the remote party should expect to receive; the remote party will have - to accept these parameters, with no option to choose different - values. + However, not all parameters follow this rule; for example, the + fingerprints [I-D.ietf-mmusic-4572-update] sent to a remote party are + calculated based on the local certificate(s) offered; the remote + party MUST either accept these parameters or reject them altogether, + with no option to choose different values. In addition, various RFCs put different conditions on the format of offers versus answers. For example, an offer may propose an - arbitrary number of media streams (i.e. m= sections), but an answer - must contain the exact same number as the offer. + arbitrary number of m= sections (i.e., media descriptions as + described in [RFC4566], Section 5.14), but an answer must contain the + exact same number as the offer. Lastly, while the exact media parameters are only known only after an offer and an answer have been exchanged, it is possible for the offerer to receive media after they have sent an offer and before they have received an answer. To properly process incoming media in this case, the offerer's media handler must be aware of the details of the offer before the answer arrives. Therefore, in order to handle session descriptions properly, the user agent needs: @@ -448,23 +452,23 @@ manipulations. Note that most applications should be able to treat the SessionDescriptions produced and consumed by these various API calls as opaque blobs; that is, the application will not need to read or change them. 3.4. Session Description Control In order to give the application control over various common session - parameters, JSEP provides control surfaces which tell the browser how - to generate session descriptions. This avoids the need for - Javascript to modify session descriptions in most cases. + parameters, JSEP provides control surfaces which tell the JSEP + implementation how to generate session descriptions. This avoids the + need for Javascript to modify session descriptions in most cases. Changes to these objects result in changes to the session descriptions generated by subsequent createOffer/Answer calls. 3.4.1. RtpTransceivers RtpTransceivers allow the application to control the RTP media associated with one m= section. Each RtpTransceiver has an RtpSender and an RtpReceiver, which an application can use to control the sending and receiving of RTP media. The application may also modify @@ -507,32 +511,32 @@ 3.5. ICE 3.5.1. ICE Gathering Overview JSEP gathers ICE candidates as needed by the application. Collection of ICE candidates is referred to as a gathering phase, and this is triggered either by the addition of a new or recycled m= section to the local session description, or new ICE credentials in the description, indicating an ICE restart. Use of new ICE credentials can be triggered explicitly by the application, or implicitly by the - browser in response to changes in the ICE configuration. + JSEP implementation in response to changes in the ICE configuration. When the ICE configuration changes in a way that requires a new gathering phase, a 'needs-ice-restart' bit is set. When this bit is set, calls to the createOffer API will generate new ICE credentials. This bit is cleared by a call to the setLocalDescription API with new ICE credentials from either an offer or an answer, i.e., from either a local- or remote-initiated ICE restart. - When a new gathering phase starts, the ICE Agent will notify the + When a new gathering phase starts, the ICE agent will notify the application that gathering is occurring through an event. Then, when - each new ICE candidate becomes available, the ICE Agent will supply + each new ICE candidate becomes available, the ICE agent will supply it to the application via an additional event; these candidates will also automatically be added to the current and/or pending local session description. Finally, when all candidates have been gathered, an event will be dispatched to signal that the gathering process is complete. Note that gathering phases only gather the candidates needed by new/recycled/restarting m= sections; other m= sections continue to use their existing candidates. Also, when bundling is active, candidates are only gathered (and exchanged) for the m= sections @@ -554,21 +558,21 @@ JSEP supports optional candidate trickling by providing APIs, as described above, that provide control and feedback on the ICE candidate gathering process. Applications that support candidate trickling can send the initial offer immediately and send individual candidates when they get the notified of a new candidate; applications that do not support this feature can simply wait for the indication that gathering is complete, and then create and send their offer, with all the candidates, at this time. Upon receipt of trickled candidates, the receiving application will - supply them to its ICE Agent. This triggers the ICE Agent to start + supply them to its ICE agent. This triggers the ICE agent to start using the new remote candidates for connectivity checks. 3.5.2.1. ICE Candidate Format In JSEP, ICE candidates are abstracted by an IceCandidate object, and as with session descriptions, SDP syntax is used for the internal representation. The candidate details are specified in an IceCandidate field, using the same SDP syntax as the "candidate-attribute" field defined in @@ -596,168 +600,152 @@ answerer when interacting with a non-JSEP endpoint that does not support the MID attribute, as discussed in Section 5.9 below). If the MID field is present in a received IceCandidate, it MUST be used for identification; otherwise, the m= section index is used instead. When creating an IceCandidate object, JSEP implementations MUST populate all of these fields. 3.5.3. ICE Candidate Policy - Typically, when gathering ICE candidates, the browser will gather all - possible forms of initial candidates - host, server reflexive, and - relay. However, in certain cases, applications may want to have more - specific control over the gathering process, due to privacy or - related concerns. For example, one may want to only use relay - candidates, to leak as little location information as possible + Typically, when gathering ICE candidates, the JSEP implementation + will gather all possible forms of initial candidates - host, server + reflexive, and relay. However, in certain cases, applications may + want to have more specific control over the gathering process, due to + privacy or related concerns. For example, one may want to only use + relay candidates, to leak as little location information as possible (keeping in mind that this choice comes with corresponding operational costs). To accomplish this, JSEP allows the application to restrict which ICE candidates are used in a session. Note that - this filtering is applied on top of any restrictions the browser - chooses to enforce regarding which IP addresses are permitted for the - application, as discussed in [I-D.ietf-rtcweb-ip-handling]. + this filtering is applied on top of any restrictions the + implementation chooses to enforce regarding which IP addresses are + permitted for the application, as discussed in + [I-D.ietf-rtcweb-ip-handling]. There may also be cases where the application wants to change which types of candidates are used while the session is active. A prime example is where a callee may initially want to use only relay candidates, to avoid leaking location information to an arbitrary caller, but then change to use all candidates (for lower operational cost) once the user has indicated they want to take the call. For - this scenario, the browser MUST allow the candidate policy to be - changed in mid-session, subject to the aforementioned interactions - with local policy. + this scenario, the JSEP implementation MUST allow the candidate + policy to be changed in mid-session, subject to the aforementioned + interactions with local policy. - To administer the ICE candidate policy, the browser will determine - the current setting at the start of each gathering phase. Then, - during the gathering phase, the browser MUST NOT expose candidates - disallowed by the current policy to the application, use them as the - source of connectivity checks, or indirectly expose them via other - fields, such as the raddr/rport attributes for other ICE candidates. - Later, if a different policy is specified by the application, the - application can apply it by kicking off a new gathering phase via an - ICE restart. + To administer the ICE candidate policy, the JSEP implementation will + determine the current setting at the start of each gathering phase. + Then, during the gathering phase, the implementation MUST NOT expose + candidates disallowed by the current policy to the application, use + them as the source of connectivity checks, or indirectly expose them + via other fields, such as the raddr/rport attributes for other ICE + candidates. Later, if a different policy is specified by the + application, the application can apply it by kicking off a new + gathering phase via an ICE restart. 3.5.4. ICE Candidate Pool - JSEP applications typically inform the browser to begin ICE gathering - via the information supplied to setLocalDescription, as this is where - the app specifies the number of media streams, and thereby ICE - components, for which to gather candidates. However, to accelerate - cases where the application knows the number of ICE components to use - ahead of time, it may ask the browser to gather a pool of potential - ICE candidates to help ensure rapid media setup. + JSEP applications typically inform the JSEP implementation to begin + ICE gathering via the information supplied to setLocalDescription, as + this is where the app specifies the number of media streams, and + thereby ICE components, for which to gather candidates. However, to + accelerate cases where the application knows the number of ICE + components to use ahead of time, it may ask the implementation to + gather a pool of potential ICE candidates to help ensure rapid media + setup. - When setLocalDescription is eventually called, and the browser goes - to gather the needed ICE candidates, it SHOULD start by checking if - any candidates are available in the pool. If there are candidates in - the pool, they SHOULD be handed to the application immediately via - the ICE candidate event. If the pool becomes depleted, either - because a larger-than-expected number of ICE components is used, or - because the pool has not had enough time to gather candidates, the - remaining candidates are gathered as usual. This only occurs for the - first offer/answer exchange, after which the candidate pool is - emptied and no longer used. + When setLocalDescription is eventually called, and the JSEP + implementation goes to gather the needed ICE candidates, it SHOULD + start by checking if any candidates are available in the pool. If + there are candidates in the pool, they SHOULD be handed to the + application immediately via the ICE candidate event. If the pool + becomes depleted, either because a larger-than-expected number of ICE + components is used, or because the pool has not had enough time to + gather candidates, the remaining candidates are gathered as usual. + This only occurs for the first offer/answer exchange, after which the + candidate pool is emptied and no longer used. One example of where this concept is useful is an application that expects an incoming call at some point in the future, and wants to minimize the time it takes to establish connectivity, to avoid clipping of initial media. By pre-gathering candidates into the pool, it can exchange and start sending connectivity checks from these candidates almost immediately upon receipt of a call. Note though that by holding on to these pre-gathered candidates, which will be kept alive as long as they may be needed, the application will consume resources on the STUN/TURN servers it is using. 3.6. Video Size Negotiation Video size negotiation is the process through which a receiver can use the "a=imageattr" SDP attribute [RFC6236] to indicate what video frame sizes it is capable of receiving. A receiver may have hard - limits on what its video decoder can process, or it may wish to - constrain what it receives due to application preferences, e.g. a - specific size for the window in which the video will be displayed. + limits on what its video decoder can process, or it may have some + maximum set by policy. Note that certain codecs support transmission of samples with aspect ratios other than 1.0 (i.e., non-square pixels). JSEP implementations will not transmit non-square pixels, but SHOULD receive and render such video with the correct aspect ratio. However, sample aspect ratio has no impact on the size negotiation described below; all dimensions are measured in pixels, whether square or not. 3.6.1. Creating an imageattr Attribute - In order to determine the limits on what video resolution a receiver - wants to receive, it will intersect its decoder hard limits with any - mandatory constraints that have been applied to the associated - MediaStreamTrack. If the decoder limits are unknown, e.g. when using - a software decoder, the mandatory constraints are used directly. For - the answerer, these mandatory constraints can be applied to the - remote MediaStreamTracks that are created by a setRemoteDescription - call, and will affect the output of the ensuing createAnswer call. - Any constraints set after setLocalDescription is used to set the - answer will result in a new offer-answer exchange. For the offerer, - because it does not know about any remote MediaStreamTracks until it - receives the answer, the offer can only reflect decoder hard limits. - If the offerer wishes to set mandatory constraints on video - resolution, it must do so after receiving the answer, and the result - will be a new offer-answer to communicate them. - - If there are no known decoder limits or mandatory constraints, the - "a=imageattr" attribute SHOULD be omitted. + The receiver will first intersect any known local limits (e.g., + hardware decoder capababilities, local policy) to determine the + absolute minimum and maximum sizes it can receive. If there are no + known local limits, the "a=imageattr" attribute SHOULD be omitted. Otherwise, an "a=imageattr" attribute is created with "recv" - direction, and the resulting resolution space formed by intersecting - the decoder limits and constraints is used to specify its minimum and + direction, and the resulting resolution space formed from the + aforementioned intersection is used to specify its minimum and maximum x= and y= values. If the intersection is the null set, i.e., - there are no resolutions that are permitted by both the decoder and - the mandatory constraints, this MUST be represented by x=0 and y=0 - values. + the degenerate case of no permitted resolutions, this MUST be + represented by x=0 and y=0 values. The rules here express a single set of preferences, and therefore, the "a=imageattr" q= value is not important. It SHOULD be set to 1.0. The "a=imageattr" field is payload type specific. When all video codecs supported have the same capabilities, use of a single attribute, with the wildcard payload type (*), is RECOMMENDED. - However, when the supported video codecs have differing capabilities, + However, when the supported video codecs have different limitations, specific "a=imageattr" attributes MUST be inserted for each payload type. As an example, consider a system with a multiformat video decoder, - which is capable of decoding any resolution from 48x48 to 720p, and - where the application has constrained the received track to at most - 360p. In this case, the implementation would generate this - attribute: + which is capable of decoding any resolution from 48x48 to 720p, In + this case, the implementation would generate this attribute: - a=imageattr:* recv [x=[48:640],y=[48:360],q=1.0] + a=imageattr:* recv [x=[48:1280],y=[48:720],q=1.0] This declaration indicates that the receiver is capable of decoding - any image resolution from 48x48 up to 640x360 pixels. + any image resolution from 48x48 up to 1280x720 pixels. 3.6.2. Interpreting an imageattr Attribute [RFC6236] defines "a=imageattr" to be an advisory field. This means that it does not absolutely constrain the video formats that the sender can use, but gives an indication of the preferred values. This specification prescribes more specific behavior. When a sender of a given MediaStreamTrack, which is producing video of a certain resolution, receives an "a=imageattr recv" attribute, it MUST check to see if the original resolution meets the size criteria specified in the attribute, and adapt the resolution accordingly by scaling (if appropriate). Note that when considering a MediaStreamTrack that is producing rotated video, the unrotated resolution MUST be used. This is required regardless of whether the receiver supports performing - receive-side rotation (e.g., through CVO), as it significantly - simplifies the matching logic. + receive-side rotation (e.g., through CVO [TS26.114]), as it + significantly simplifies the matching logic. For the purposes of resolution negotiation, only size limits are considered. Any other values, e.g. picture or sample aspect ratio, MUST be ignored. When communicating with a non-JSEP endpoint, multiple relevant "a=imageattr recv" attributes may be present in a received m= section. If this occurs, attributes other than the one with the highest "q=" value MUST be ignored. If multiple attributes have the same "q=" value, those that appear after the first such attribute in @@ -944,27 +932,27 @@ 4.1.1. Constructor The PeerConnection constructor allows the application to specify global parameters for the media session, such as the STUN/TURN servers and credentials to use when gathering candidates, as well as the initial ICE candidate policy and pool size, and also the bundle policy to use. If an ICE candidate policy is specified, it functions as described in - Section 3.5.3, causing the browser to only surface the permitted - candidates (including any internal browser filtering) to the - application, and only use those candidates for connectivity checks. - The set of available policies is as follows: + Section 3.5.3, causing the JSEP implementation to only surface the + permitted candidates (including any implementation-internal + filtering) to the application, and only use those candidates for + connectivity checks. The set of available policies is as follows: - all: All candidates permitted by browser policy will be gathered and - used. + all: All candidates permitted by implementation policy will be + gathered and used. relay: All candidates except relay candidates will be filtered out. This obfuscates the location information that might be ascertained by the remote peer from the received candidates. Depending on how the application deploys and chooses relay servers, this could obfuscate location to a metro or possibly even global level. The default ICE candidate policy MUST be set to "all" as this is generally the desired policy, and also typically reduces use of application TURN server resources significantly. @@ -973,71 +961,74 @@ number of ICE components to pre-gather candidates for. Because pre- gathering results in utilizing STUN/TURN server resources for potentially long periods of time, this must only occur upon application request, and therefore the default candidate pool size MUST be zero. The application can specify its preferred policy regarding use of bundle, the multiplexing mechanism defined in [I-D.ietf-mmusic-sdp-bundle-negotiation]. Regardless of policy, the application will always try to negotiate bundle onto a single - transport, and will offer a single bundle group across all media - section; use of this single transport is contingent upon the answerer - accepting bundle. However, by specifying a policy from the list - below, the application can control exactly how aggressively it will - try to bundle media streams together, which affects how it will + transport, and will offer a single bundle group across all m= + sections; use of this single transport is contingent upon the + answerer accepting bundle. However, by specifying a policy from the + list below, the application can control exactly how aggressively it + will try to bundle media streams together, which affects how it will interoperate with a non-bundle-aware endpoint. When negotiating with a non-bundle-aware endpoint, only the streams not marked as bundle- only streams will be established. The set of available policies is as follows: - balanced: The first media section of each type (audio, video, or + balanced: The first m= section of each type (audio, video, or application) will contain transport parameters, which will allow an answerer to unbundle that section. The second and any - subsequent media section of each type will be marked bundle-only. + subsequent m= section of each type will be marked bundle-only. The result is that if there are N distinct media types, then candidates will be gathered for for N media streams. This policy balances desire to multiplex with the need to ensure basic audio and video can still be negotiated in legacy cases. When acting as answerer, if there is no bundle group in the offer, the implementation will reject all but the first m= section of each type. - max-compat: All media sections will contain transport parameters; - none will be marked as bundle-only. This policy will allow all - streams to be received by non-bundle-aware endpoints, but require - separate candidates to be gathered for each media stream. + max-compat: All m= sections will contain transport parameters; none + will be marked as bundle-only. This policy will allow all streams + to be received by non-bundle-aware endpoints, but require separate + candidates to be gathered for each media stream. - max-bundle: Only the first media section will contain transport + max-bundle: Only the first m= section will contain transport parameters; all streams other than the first will be marked as bundle-only. This policy aims to minimize candidate gathering and maximize multiplexing, at the cost of less compatibility with legacy endpoints. When acting as answerer, the implementation will reject any m= sections other than the first m= section, unless they are in the same bundle group as that m= section. As it provides the best tradeoff between performance and compatibility with legacy endpoints, the default bundle policy MUST be set to "balanced". The application can specify its preferred policy regarding use of RTP/RTCP multiplexing [RFC5761] using one of the following policies: - negotiate: The browser will gather both RTP and RTCP candidates but - also will offer "a=rtcp-mux", thus allowing for compatibility with - either multiplexing or non-multiplexing endpoints. + negotiate: The JSEP implementation will gather both RTP and RTCP + candidates but also will offer "a=rtcp-mux", thus allowing for + compatibility with either multiplexing or non-multiplexing + endpoints. - require: The browser will only gather RTP candidates. This halves - the number of candidates that the offerer needs to gather. - Applying a description with an m= section that does not contain an - "a=rtcp-mux" attribute will cause an error to be returned. + require: The JSEP implementation will only gather RTP candidates and + will insert an "a=rtcp-mux-only" indication into any new m= + sections in offers it generates. This halves the number of + candidates that the offerer needs to gather. Applying a + description with an m= section that does not contain an "a=rtcp- + mux" attribute will cause an error to be returned. The default multiplexing policy MUST be set to "require". Implementations MAY choose to reject attempts by the application to set the multiplexing policy to "negotiate". 4.1.2. addTrack The addTrack method adds a MediaStreamTrack to the PeerConnection, using the MediaStream argument to associate the track with other tracks in the same MediaStream, so that they can be added to the same @@ -1048,21 +1039,21 @@ call to setRemoteDescription() and does not have a local track. Otherwise, a new transceiver will be created, as described in Section 4.1.4. 4.1.3. removeTrack The removeTrack method removes a MediaStreamTrack from the PeerConnection, using the RtpSender argument to indicate which sender should have its track removed. The sender's track is cleared, and the sender stops sending. Future calls to createOffer will mark the - media description associated with the sender as recvonly (if + m= section associated with the sender as recvonly (if transceiver.currentDirection is sendrecv) or as inactive (if transceiver.currentDirection is sendonly). 4.1.4. addTransceiver The addTransceiver method adds a new RtpTransceiver to the PeerConnection. If a MediaStreamTrack argument is provided, then the transceiver will be configured with that media type and the track will be attached to the transceiver. Otherwise, the application MUST explicitly specify the type; this mode is useful for creating @@ -1087,21 +1078,21 @@ The createDataChannel method also includes a number of arguments which are used by the PeerConnection (e.g., maxPacketLifetime) but are not reflected in the SDP and do not affect the JSEP state. 4.1.6. createOffer The createOffer method generates a blob of SDP that contains a [RFC3264] offer with the supported configurations for the session, including descriptions of the media added to this PeerConnection, the codec/RTP/RTCP options supported by this implementation, and any - candidates that have been gathered by the ICE Agent. An options + candidates that have been gathered by the ICE agent. An options parameter may be supplied to provide additional control over the generated offer. This options parameter allows an application to trigger an ICE restart, for the purpose of reestablishing connectivity. In the initial offer, the generated SDP will contain all desired functionality for the session (functionality that is supported but not desired by default may be omitted); for each SDP line, the generation of the SDP will follow the process defined for generating an initial offer from the document that specifies the given SDP line. @@ -1122,51 +1113,55 @@ exact handling of subsequent offer generation is detailed in Section 5.2.2. below. Session descriptions generated by createOffer must be immediately usable by setLocalDescription; if a system has limited resources (e.g. a finite number of decoders), createOffer should return an offer that reflects the current state of the system, so that setLocalDescription will succeed when it attempts to acquire those resources. - Calling this method may do things such as generate new ICE + Calling this method may do things such as generating new ICE credentials, but does not result in candidate gathering, or cause - media to start or stop flowing. + media to start or stop flowing. Specifically, the offer is not + applied, and does not become the pending local description, until + setLocalDescription is called. 4.1.7. createAnswer The createAnswer method generates a blob of SDP that contains a [RFC3264] SDP answer with the supported configuration for the session that is compatible with the parameters supplied in the most recent call to setRemoteDescription, which MUST have been called prior to calling createAnswer. Like createOffer, the returned blob contains descriptions of the media added to this PeerConnection, the codec/RTP/RTCP options negotiated for this session, and any - candidates that have been gathered by the ICE Agent. An options + candidates that have been gathered by the ICE agent. An options parameter may be supplied to provide additional control over the generated answer. As an answer, the generated SDP will contain a specific configuration that specifies how the media plane should be established; for each SDP line, the generation of the SDP must follow the process defined for generating an answer from the document that specifies the given SDP line. The exact handling of answer generation is detailed in Section 5.3. below. Session descriptions generated by createAnswer must be immediately usable by setLocalDescription; like createOffer, the returned description should reflect the current state of the system. - Calling this method may do things such as generate new ICE - credentials, but does not trigger candidate gathering or change media - state. + Calling this method may do things such as generating new ICE + credentials, but does not trigger candidate gathering or cause a + media state change. Specifically, the answer is not applied, and + does not become the pending local description, until + setLocalDescription is called. 4.1.8. SessionDescriptionType Session description objects (RTCSessionDescription) may be of type "offer", "pranswer", "answer" or "rollback". These types provide information as to how the description parameter should be parsed, and how the media state should be changed. "offer" indicates that a description should be parsed as an offer; said description may include many possible media configurations. A @@ -1198,48 +1193,55 @@ answers as provisional answers, and only apply an answer as final when it receives one that meets its criteria (e.g. a live user instead of voicemail). "rollback" is a special session description type implying that the state machine should be rolled back to the previous stable state, as described in Section 4.1.8.2. The contents MUST be empty. 4.1.8.1. Use of Provisional Answers - Most web applications will not need to create answers using the + Most applications will not need to create answers using the "pranswer" type. While it is good practice to send an immediate - response to an "offer", in order to warm up the session transport and - prevent media clipping, the preferred handling for a web application - would be to create and send an "inactive" final answer immediately - after receiving the offer. Later, when the called user actually - accepts the call, the application can create a new "sendrecv" offer - to update the previous offer/answer pair and start the media flow. - While this could also be done with an inactive "pranswer", followed - by a sendrecv "answer", the initial "pranswer" leaves the offer- - answer exchange open, which means that neither side can send an - updated offer during this time. + response to an offer, in order to warm up the session transport and + prevent media clipping, the preferred handling for a JSEP application + is to create and send a "sendonly" final answer with a null + MediaStreamTrack immediately after receiving the offer, which will + prevent media from being sent by the caller, and allow media to be + sent immediately upon answer by the callee. Later, when the callee + actually accepts the call, the application can plug in the real + MediaStreamTrack and create a new "sendrecv" offer to update the + previous offer/answer pair and start bidirectional media flow. While + this could also be done with a "sendonly" pranswer, followed by a + "sendrecv" answer, the initial pranswer leaves the offer-answer + exchange open, which means that the caller cannot send an updated + offer during this time. - As an example, consider a typical web application that will set up a - data channel, an audio channel, and a video channel. When an - endpoint receives an offer with these channels, it could send an - answer accepting the data channel for two-way data, and accepting the - audio and video tracks as inactive or receive-only. It could then - ask the user to accept the call, acquire the local media streams, and - send a new offer to the remote side moving the audio and video to be - two-way media. By the time the human has accepted the call and - triggered the new offer, it is likely that the ICE and DTLS - handshaking for all the channels will already have finished. + As an example, consider a typical JSEP application that wants to set + up audio and video as quickly as possible. When the callee receives + an offer with audio and video MediaStreamTracks, it will send an + immediate answer accepting these tracks as sendonly (meaning that the + caller will not send the callee any media yet, and because the callee + has not yet added its own MediaStreamTracks, the callee will not send + any media either). It will then ask the user to accept the call and + acquire the needed local tracks. Upon acceptance by the user, the + application will plug in the tracks it has acquired, which, because + ICE and DTLS handshaking have likely completed by this point, can + start transmitting immediately. The application will also send a new + offer to the remote side indicating call acceptance and moving the + audio and video to be two-way media. A detailed example flow along + these lines is shown in Section 7.3. Of course, some applications may not be able to perform this double offer-answer exchange, particularly ones that are attempting to - gateway to legacy signaling protocols. In these cases, "pranswer" - can still provide the application with a mechanism to warm up the + gateway to legacy signaling protocols. In these cases, pranswer can + still provide the application with a mechanism to warm up the transport. 4.1.8.2. Rollback In certain situations it may be desirable to "undo" a change made to setLocalDescription or setRemoteDescription. Consider a case where a call is ongoing, and one side wants to change some of the session parameters; that side generates an updated offer and then calls setLocalDescription. However, the remote side, either before or after setRemoteDescription, decides it does not want to accept the @@ -1329,32 +1331,32 @@ If setLocalDescription was previously called with an offer, and setRemoteDescription is called with an answer (provisional or final), and the media directions are compatible, and media is available to send, this will result in the starting of media transmission. 4.1.11. currentLocalDescription The currentLocalDescription method returns the current negotiated local description - i.e., the local description from the last successful offer/answer exchange - in addition to any local - candidates that have been generated by the ICE Agent since the local + candidates that have been generated by the ICE agent since the local description was set. A null object will be returned if an offer/answer exchange has not yet been completed. 4.1.12. pendingLocalDescription The pendingLocalDescription method returns a copy of the local description currently in negotiation - i.e., a local offer set without any corresponding remote answer - in addition to any local - candidates that have been generated by the ICE Agent since the local + candidates that have been generated by the ICE agent since the local description was set. A null object will be returned if the state of the PeerConnection is "stable" or "have-remote-offer". 4.1.13. currentRemoteDescription The currentRemoteDescription method returns a copy of the current negotiated remote description - i.e., the remote description from the last successful offer/answer exchange - in addition to any remote @@ -1437,37 +1439,37 @@ if decreased, the now-superfluous candidates are discarded. o The bundle and RTCP-multiplexing policies MUST NOT be changed after the construction of the PeerConnection. This call may result in a change to the state of the ICE Agent. 4.1.17. addIceCandidate The addIceCandidate method provides a remote candidate to the ICE - Agent, which, if parsed successfully, will be added to the current + agent, which, if parsed successfully, will be added to the current and/or pending remote description according to the rules defined for Trickle ICE. The pair of MID and ufrag is used to determine the m= section and ICE candidate generation to which the candidate belongs. If the MID is not present, the m= section index is used to look up the locally generated MID (see Section 5.9), which is used in place of a supplied MID. If these values or the candidate string are invalid, an error is generated. The purpose of the ufrag is to resolve ambiguities when trickle ICE is in progress during an ICE restart. If the ufrag is absent, the candidate MUST be assumed to belong to the most recently applied remote description. Connectivity checks will be sent to the new candidate. This method can also be used to provide an end-of-candidates - indication to the ICE Agent, as defined in [I-D.ietf-ice-trickle]). + indication to the ICE agent, as defined in [I-D.ietf-ice-trickle]). The MID and ufrag are used as described above to determine the m= section and ICE generation for which candidate gathering is complete. If the ufrag is not present, then the end-of-candidates indication MUST be assumed to apply to the relevant m= section in the most recently applied remote description. If neither the MID nor the m= index is present, then the indication MUST be assumed to apply to all m= sections in the most recently applied remote description. This call will result in a change to the state of the ICE Agent, and may result in a change to media state if it results in connectivity @@ -1545,166 +1547,95 @@ excluded by subsequent calls to createOffer and createAnswer, in which case the corresponding media formats in the associated m= section will be excluded. The codec preferences cannot add media formats that would otherwise not be present. This includes codecs that were not negotiated in a previous offer/answer exchange that included the transceiver. The codec preferences of an RtpTransceiver can also determine the order of codecs in subsequent calls to createOffer and createAnswer, in which case the order of the media formats in the associated m= - section will match. However, the codec preferences cannot change the - order of the media formats after an answer containing the transceiver - has been applied. At this point, codecs can only be removed, not - reordered. + section will follow the specified preferences. 5. SDP Interaction Procedures This section describes the specific procedures to be followed when creating and parsing SDP objects. 5.1. Requirements Overview JSEP implementations must comply with the specifications listed below that govern the creation and processing of offers and answers. - The first set of specifications is the "mandatory-to-implement" set. - All implementations must support these behaviors, but may not use all - of them if the remote side, which may not be a JSEP endpoint, does - not support them. - - The second set of specifications is the "mandatory-to-use" set. The - local JSEP endpoint and any remote endpoint must indicate support for - these specifications in their session descriptions. - -5.1.1. Implementation Requirements - - Implementations of JSEP MUST conform to [I-D.ietf-rtcweb-rtp-usage]. - This list of mandatory-to-implement specifications is derived from - the requirements outlined in that document and from - [I-D.ietf-rtcweb-security-arch]. - - R-1 [RFC4566] is the base SDP specification and MUST be - implemented. - - R-2 [RFC5764] MUST be supported for signaling the UDP/TLS/RTP/SAVPF - [RFC5764], TCP/DTLS/RTP/SAVPF [RFC7850], "UDP/DTLS/SCTP" - [I-D.ietf-mmusic-sctp-sdp], and "TCP/DTLS/SCTP" - [I-D.ietf-mmusic-sctp-sdp] RTP profiles. - - R-3 [RFC5245] MUST be implemented for signaling the ICE credentials - and candidate lines corresponding to each media stream. The - ICE implementation MUST be a Full implementation, not a Lite - implementation. - - R-4 [RFC5763] MUST be implemented to signal DTLS certificate - fingerprints. - - R-5 [RFC5888] MUST be implemented for signaling grouping - information, and MUST be used to identify m= lines via the - a=mid attribute. - - R-6 [I-D.ietf-mmusic-msid] MUST be supported, in order to signal - associations between RTP objects and W3C MediaStreams and - MediaStreamTracks in a standard way. - - R-7 The bundle mechanism in - [I-D.ietf-mmusic-sdp-bundle-negotiation] MUST be supported to - signal the ability to multiplex RTP streams on a single UDP - port, in order to avoid excessive use of port number resources. - - R-8 The SDP attributes of "sendonly", "recvonly", "inactive", and - "sendrecv" from [RFC4566] MUST be implemented to signal - information about media direction. - - R-9 [RFC5576] MUST be implemented to signal RTP SSRC values and - grouping semantics. - - R-10 [RFC4585] MUST be implemented to signal RTCP based feedback. - - R-11 [RFC5761] MUST be implemented to signal multiplexing of RTP and - RTCP. - - R-12 [RFC5506] MUST be implemented to signal reduced-size RTCP - messages. - - R-13 [RFC4588] MUST be implemented to signal RTX payload type - associations. - - R-14 [RFC3556] MUST be supported for control of RTCP bandwidth - limits. - - The SDES SRTP keying mechanism from [RFC4568] MUST NOT be - implemented, as discussed in [I-D.ietf-rtcweb-security-arch]. - - As required by [RFC4566], Section 5.13, JSEP implementations MUST - ignore unknown attribute (a=) lines. - -5.1.2. Usage Requirements +5.1.1. Usage Requirements - All session descriptions handled by JSEP endpoints, both local and - remote, MUST indicate support for the following specifications. If - any of these are absent, this omission MUST be treated as an error. + All session descriptions handled by JSEP implementations, both local + and remote, MUST indicate support for the following specifications. + If any of these are absent, this omission MUST be treated as an + error. - U-1 ICE, as specified in [RFC5245], MUST be used. Note that the + o ICE, as specified in [RFC5245], MUST be used. Note that the remote endpoint may use a Lite implementation; implementations MUST properly handle remote endpoints which do ICE-Lite. - U-2 DTLS [RFC6347] or DTLS-SRTP [RFC5763], MUST be used, as + o DTLS [RFC6347] or DTLS-SRTP [RFC5763], MUST be used, as appropriate for the media type, as specified in [I-D.ietf-rtcweb-security-arch] -5.1.3. Profile Names and Interoperability + The SDES SRTP keying mechanism from [RFC4568] MUST NOT be used, as + discussed in [I-D.ietf-rtcweb-security-arch]. - For media m= sections, JSEP endpoints MUST support both the "UDP/TLS/ - RTP/SAVPF" and "TCP/DTLS/RTP/SAVPF" profiles and MUST indicate one of - these two profiles for each media m= line they produce in an offer. - For data m= sections, JSEP endpoints must support both the "UDP/DTLS/ - SCTP" and "TCP/DTLS/SCTP" profiles and MUST indicate one of these two - profiles for each data m= line they produce in an offer. Because ICE - can select either TCP or UDP transport depending on network - conditions, both advertisements are consistent with ICE eventually - selecting either either UDP or TCP. +5.1.2. Profile Names and Interoperability + + For media m= sections, JSEP implementations MUST support the + "UDP/TLS/RTP/SAVPF" profile specified in [RFC7850], and MUST indicate + this profile for each media m= line they produce in an offer. For + data m= sections, implementations MUST support the "UDP/DTLS/SCTP" + profile and MUST indicate this profile for each data m= line they + produce in an offer. Because ICE can select either UDP [RFC5245] or + TCP [RFC6544] transport depending on network conditions, this + advertisement is consistent with ICE eventually selecting either + either UDP or TCP. Unfortunately, in an attempt at compatibility, some endpoints generate other profile strings even when they mean to support one of these profiles. For instance, an endpoint might generate "RTP/AVP" but supply "a=fingerprint" and "a=rtcp-fb" attributes, indicating its willingness to support "(UDP,TCP)/TLS/RTP/SAVPF". In order to - simplify compatibility with such endpoints, JSEP endpoints MUST + simplify compatibility with such endpoints, JSEP implementations MUST follow the following rules when processing the media m= sections in an offer: o The profile in any "m=" line in any answer MUST exactly match the profile provided in the offer. o Any profile matching the following patterns MUST be accepted: "RTP/[S]AVP[F]" and "(UDP/TCP)/TLS/RTP/SAVP[F]" o Because DTLS-SRTP is REQUIRED, the choice of SAVP or AVP has no effect; support for DTLS-SRTP is determined by the presence of one or more "a=fingerprint" attribute. Note that lack of an "a=fingerprint" attribute will lead to negotiation failure. o The use of AVPF or AVP simply controls the timing rules used for RTCP feedback. If AVPF is provided, or an "a=rtcp-fb" attribute is present, assume AVPF timing, i.e., a default value of "trr- int=0". Otherwise, assume that AVPF is being used in an AVP compatible mode and use a value of "trr-int=4000". - o For data m= sections, JSEP endpoints MUST support receiving the + o For data m= sections, implementations MUST support receiving the "UDP/DTLS/SCTP", "TCP/DTLS/SCTP", or "DTLS/SCTP" (for backwards compatibility) profiles. - Note that re-offers by JSEP endpoints MUST use the correct profile - strings even if the initial offer/answer exchange used an (incorrect) - older profile string. + Note that re-offers by JSEP implementations MUST use the correct + profile strings even if the initial offer/answer exchange used an + (incorrect) older profile string. 5.2. Constructing an Offer When createOffer is called, a new SDP description must be created that includes the functionality specified in [I-D.ietf-rtcweb-rtp-usage]. The exact details of this process are explained below. 5.2.1. Initial Offers @@ -1744,20 +1675,23 @@ o Encryption Keys ("k=") lines do not provide sufficient security and MUST NOT be included. o A "t=" line MUST be added, as specified in [RFC4566], Section 5.9; both and SHOULD be set to zero, e.g. "t=0 0". o An "a=ice-options" line with the "trickle" option MUST be added, as specified in [I-D.ietf-ice-trickle], Section 4. + o If WebRTC identity is being used, an "a=identity" line as + described in [I-D.ietf-rtcweb-security-arch], Section 5. + The next step is to generate m= sections, as specified in [RFC4566] Section 5.14. An m= section is generated for each RtpTransceiver that has been added to the PeerConnection, excluding any stopped RtpTransceivers. This is done in the order the RtpTransceivers were added to the PeerConnection. For each m= section generated for an RtpTransceiver, establish a mapping between the transceiver and the index of the generated m= section. @@ -1821,64 +1755,70 @@ occurs when the description is applied. The generated MID value can be considered a "proposed" MID at this point. o A direction attribute which is the same as that of the associated transceiver. o For each media format on the m= line, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6, and [RFC3264], Section 5.1. - o If this m= section is for media with configurable durations of - media per packet, e.g., audio, an "a=maxptime" line, indicating - the maximum amount of media, specified in milliseconds, that can - be encapsulated in each packet, as specified in [RFC4566], - Section 6. This value is set to the smallest of the maximum - duration values across all the codecs included in the m= section. - - o If this m= section is for video media, and there are known - limitations on the size of images which can be decoded, an - "a=imageattr" line, as specified in Section 3.6. - o For each primary codec where RTP retransmission should be used, a corresponding "a=rtpmap" line indicating "rtx" with the clock rate of the primary codec and an "a=fmtp" line that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. o For each supported FEC mechanism, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are specified in [I-D.ietf-rtcweb-fec], Section 6, and specific usage for each media type is outlined in Sections 4 and 5. + o If this m= section is for media with configurable durations of + media per packet, e.g., audio, an "a=maxptime" line, indicating + the maximum amount of media, specified in milliseconds, that can + be encapsulated in each packet, as specified in [RFC4566], + Section 6. This value is set to the smallest of the maximum + duration values across all the codecs included in the m= section. + + o If this m= section is for video media, and there are known + limitations on the size of images which can be decoded, an + "a=imageattr" line, as specified in Section 3.6. + o For each supported RTP header extension, an "a=extmap" line, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/MUST be supported is specified in [I-D.ietf-rtcweb-rtp-usage], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4. o For each supported RTCP feedback mechanism, an "a=rtcp-fb" mechanism, as specified in [RFC4585], Section 4.2. The list of RTCP feedback mechanisms that SHOULD/MUST be supported is specified in [I-D.ietf-rtcweb-rtp-usage], Section 5.1. - o If the bundle policy for this PeerConnection is set to "max- - bundle", and this is not the first m= section, or the bundle - policy is set to "balanced", and this is not the first m= section - for this media type, an "a=bundle-only" line. - o If the RtpTransceiver has a sendrecv or sendonly direction: - * An "a=msid" line, as specified in [I-D.ietf-mmusic-msid], - Section 2. + * For each MediaStream that was associated with the transceiver + when it was created via addTrack or addTransceiver, an "a=msid" + line, as specified in [I-D.ietf-mmusic-msid], Section 2. If a + MediaStreamTrack is attached to the transceiver's RtpSender, + the "a=msid" lines MUST use that track's ID. If no + MediaStreamTrack is attached, a valid ID MUST be generated, in + the same way that the implementation generates IDs for local + tracks. + + * If no MediaStream is associated with the transceiver, a single + "a=msid" line with the special value "-" in place of the + MediaStream ID, as specified in [I-D.ietf-mmusic-msid], + Section 3. The track ID MUST be selected as described above. o If the RtpTransceiver has a sendrecv or sendonly direction, and the application has specified RID values or has specified more than one encoding in the RtpSenders's parameters, an "a=rid" line for each encoding specified. The "a=rid" line is specified in [I-D.ietf-mmusic-rid], and its direction MUST be "send". If the application has chosen a RID value, it MUST be used as the rid- identifier; otherwise a RID value MUST be generated by the implementation. RID values MUST be generated in a fashion that does not leak user information, e.g., randomly or using a per- @@ -1888,26 +1828,30 @@ specified, or only one encoding is specified but without a RID value, then no "a=rid" lines are generated. o If the RtpTransceiver has a sendrecv or sendonly direction and more than one "a=rid" line has been generated, an "a=simulcast" line, with direction "send", as defined in [I-D.ietf-mmusic-sdp-simulcast], Section 6.2. The list of RIDs MUST include all of the RID identifiers used in the "a=rid" lines for this m= section. + o If the bundle policy for this PeerConnection is set to "max- + bundle", and this is not the first m= section, or the bundle + policy is set to "balanced", and this is not the first m= section + for this media type, an "a=bundle-only" line. + The following attributes, which are of category IDENTICAL or TRANSPORT, MUST appear only in "m=" sections which either have a unique address or which are associated with the bundle-tag. (In initial offers, this means those "m=" sections which do not contain - an "a=bundle-only" attribute. - + an "a=bundle-only" attribute.) o "a=ice-ufrag" and "a=ice-pwd" lines, as specified in [RFC5245], Section 15.4. o An "a=fingerprint" line for each of the endpoint's certificates, as specified in [RFC4572], Section 5; the digest algorithm used for the fingerprint MUST match that used in the certificate signature. o An "a=setup" line, as specified in [RFC4145], Section 4, and clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. @@ -1915,40 +1859,59 @@ o An "a=dtls-id" line, as specified in [I-D.ietf-mmusic-dtls-sdp] Section 5.2. o An "a=rtcp" line, as specified in [RFC3605], Section 2.1, containing the dummy value "9 IN IP4 0.0.0.0", because no candidates have yet been gathered. o An "a=rtcp-mux" line, as specified in [RFC5761], Section 5.1.3. + o If the RTP/RTCP multiplexing policy is "require", an "a=rtcp-mux- + only" line, as specified in [I-D.ietf-mmusic-mux-exclusive], + Section 4. + o An "a=rtcp-rsize" line, as specified in [RFC5506], Section 5. Lastly, if a data channel has been created, a m= section MUST be generated for data. The field MUST be set to "application" and the field MUST be set to "UDP/DTLS/SCTP" [I-D.ietf-mmusic-sctp-sdp]. The "fmt" value MUST be set to "webrtc- datachannel" as specified in [I-D.ietf-mmusic-sctp-sdp], Section 4.1. - Within the data m= section, the "a=mid", "a=ice-ufrag", "a=ice-pwd", - "a=fingerprint", "a=dtls-id", and "a=setup" lines MUST be included as - mentioned above, along with an "a=fmtp:webrtc-datachannel" line and - an "a=sctp-port" line referencing the SCTP port number as defined in - [I-D.ietf-mmusic-sctp-sdp], Section 4.1. + Within the data m= section, an "a=mid" line MUST be generated and + included as described above, along with an "a=sctp-port" line + referencing the SCTP port number, as defined in + [I-D.ietf-mmusic-sctp-sdp], Section 5.1, and, if appropriate, an + "a=max-message-size" line, as defined in [I-D.ietf-mmusic-sctp-sdp], + Section 6.1. + + As discussed above, the following attributes of category IDENTICAL or + TRANSPORT are included only if the data m= section either has a + unique address or is associated with the bundle-tag (e.g., if it is + the only m= section): + + o "a=ice-ufrag" + + o "a=ice-pwd" + o "a=fingerprint" + + o "a=setup" + + o "a=dtls-id" Once all m= sections have been generated, a session-level "a=group" attribute MUST be added as specified in [RFC5888]. This attribute - MUST have semantics "bundle", and MUST include the mid identifiers of - each m= section. The effect of this is that the browser offers all - m= sections as one bundle group. However, whether the m= sections - are bundle-only or not depends on the bundle policy. + MUST have semantics "BUNDLE", and MUST include the mid identifiers of + each m= section. The effect of this is that the JSEP implementation + offers all m= sections as one bundle group. However, whether the m= + sections are bundle-only or not depends on the bundle policy. The next step is to generate session-level lip sync groups as defined in [RFC5888], Section 7. For each MediaStream referenced by more than one RtpTransceiver (by passing those MediaStreams as arguments to the addTrack and addTransceiver methods), a group of type "LS" MUST be added that contains the mid values for each RtpTransceiver. Attributes which SDP permits to either be at the session level or the media level SHOULD generally be at the media level even if they are identical. This promotes readability, especially if one of a set of @@ -2023,69 +1986,73 @@ o If an RtpTransceiver is stopped and is not associated with an m= section, an m= section MUST NOT be generated for it. This prevents adding back RtpTransceivers whose m= sections were recycled and used for a new RtpTransceiver in a previous offer/ answer exchange, as described above. o If an RtpTransceiver has been stopped and is associated with an m= section, and the m= section is not being recycled as described above, an m= section MUST be generated for it with the port set to - zero and the "a=msid" line removed. + zero and all "a=msid" lines removed. - o For RtpTransceivers that are not stopped, the "a=msid" line MUST - stay the same if they are present in the current description. + o For RtpTransceivers that are not stopped, the "a=msid" line(s) + MUST stay the same if they are present in the current description, + regardless of changes to the transceiver's direction or track. If + no "a=msid" line is present in the current description, "a=msid" + line(s) MUST be generated according to the same rules as for an + initial offer. o Each "m=" and c=" line MUST be filled in with the port, protocol, and address of the default candidate for the m= section, as described in [RFC5245], Section 4.3. If ICE checking has already completed for one or more candidate pairs and a candidate pair is in active use, then that pair MUST be used, even if ICE has not yet completed. Note that this differs from the guidance in [RFC5245], Section 9.1.2.2, which only refers to offers created when ICE has completed. In each case, if no RTP candidates have yet been gathered, dummy values MUST be used, as described above. o Each "a=mid" line MUST stay the same. o Each "a=ice-ufrag" and "a=ice-pwd" line MUST stay the same, unless the ICE configuration has changed (either changes to the supported STUN/TURN servers, or the ICE candidate policy), or the "IceRestart" option ( Section 5.2.3.1 was specified. If the m= section is bundled into another m= section, it still MUST NOT contain any ICE credentials. - o If the m= section is not bundled into another m= section, an - "a=rtcp" attribute line MUST be added with of the default RTCP - candidate, as indicated in [RFC5761], Section 5.1.3. + o If the m= section is not bundled into another m= section, its + "a=rtcp" attribute line MUST be filled in with the port and + address of the default RTCP candidate, as indicated in [RFC5761], + Section 5.1.3. If no RTCP candidates have yet been gathered, + dummy values MUST be used, as described in the initial offer + section above. o If the m= section is not bundled into another m= section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an "a=candidate" line MUST be added, as defined in [RFC5245], Section 4.3., paragraph 3. If candidate gathering for the section has completed, an "a=end-of-candidates" attribute MUST be added, as described in [I-D.ietf-ice-trickle], Section 9.3. If the m= section is bundled into another m= section, both "a=candidate" and "a=end-of-candidates" MUST be omitted. - o For RtpTransceivers that are still present, the "a=msid" line MUST - stay the same. - o For RtpTransceivers that are still present, the "a=rid" lines MUST stay the same. o For RtpTransceivers that are still present, any "a=simulcast" line MUST stay the same. o If any RtpTransceiver has been stopped, the port MUST be set to - zero and the "a=msid" line MUST be removed. + zero and all "a=msid" lines MUST be removed. o If any RtpTransceiver has been added, and there exists a m= section with a zero port in the current local description or the current remote description, that m= section MUST be recycled by generating a m= section for the added RtpTransceiver as if the m= section were being added to session description, except that instead of adding it, the generated m= section replaces the m= section with a zero port. The new m= section MUST contain a new MID. @@ -2100,52 +2067,76 @@ new offer, the following adjustments are made based on the contents of the corresponding m= section in the current remote description, if any: o The m= line and corresponding "a=rtpmap" and "a=fmtp" lines MUST only include codecs present in the most recent answer which have not been excluded by the codec preferences of the associated transceiver. Note that non-JSEP endpoints are not subject to these restrictions, and might offer media formats that were not present in the most recent answer, as specified in [RFC3264], - Section 8. Therefore, JSEP endpoints MUST be prepared to receive - such offers. + Section 8. Therefore, JSEP implementations MUST be prepared to + receive such offers. - o The media formats on the m= line MUST be generated in the same - order as in the current local description. + o Unless codec preferences have been set for the associated + transceiver, the media formats on the m= line MUST be generated in + the same order as in the current local description. o The RTP header extensions MUST only include those that are present in the most recent answer. o The RTCP feedback extensions MUST only include those that are present in the most recent answer. o The "a=rtcp" line MUST only be added if the most recent answer did not include an "a=rtcp-mux" line. o The "a=rtcp-mux" line MUST only be added if present in the most recent answer. - o The "a=rtcp-mux-only" line MUST only be added if present in the - most recent answer. + o The "a=rtcp-mux-only" line MUST NOT be added. o The "a=rtcp-rsize" line MUST only be added if present in the most recent answer. - The "a=group:BUNDLE" attribute MUST include the mid identifiers + o An "a=bundle-only" line MUST NOT be added, as indicated in + [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 6. Instead, + JSEP implementations MUST simply omit parameters in the IDENTICAL + and TRANSPORT categories for bundled m= sections, as described in + [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 8.1. + + o Note that if media m= sections are bundled into a data m= section, + then certain TRANSPORT and IDENTICAL attributes may appear in the + data m= section even if they would otherwise only be appropriate + for a media m= section (e.g., "a=rtcp-mux"). This cannot happen + in initial offers because in the initial offer JSEP + implementations always list media m= sections (if any) before the + data m= section (if any), and at least one of those media m= + sections will not have the "a=bundle-only" attribute. Therefore, + in initial offers, any "a=bundle-only" m= sections will be bundled + into a preceding non-bundle-only media m= section. + + The "a=group:BUNDLE" attribute MUST include the MID identifiers specified in the bundle group in the most recent answer, minus any m= sections that have been marked as rejected, plus any newly added or re-enabled m= sections. In other words, the bundle attribute must contain all m= sections that were previously bundled, as long as they are still alive, as well as any new m= sections. - The "LS" groups are generated in the same way as with initial offers. + "a=group:LS" attributes are generated in the same way as for initial + offers, with the additional stipulation that any lip sync groups that + were present in the most recent answer MUST continue to exist and + MUST contain any previously existing MID identifiers, as long as the + identified m= sections still exist and are not rejected, and the + group still contains at least two MID identifiers. This ensures that + any synchronized "recvonly" m= sections continue to be synchronized + in the new offer. 5.2.3. Options Handling The createOffer method takes as a parameter an RTCOfferOptions object. Special processing is performed when generating a SDP description if the following options are present. 5.2.3.1. IceRestart If the "IceRestart" option is specified, with a value of "true", the @@ -2161,33 +2152,33 @@ 5.2.3.2. VoiceActivityDetection If the "VoiceActivityDetection" option is specified, with a value of "true", the offer MUST indicate support for silence suppression in the audio it receives by including comfort noise ("CN") codecs for each offered audio codec, as specified in [RFC3389], Section 5.1, except for codecs that have their own internal silence suppression support. For codecs that have their own internal silence suppression support, the appropriate fmtp parameters for that codec MUST be specified to indicate that silence suppression for received audio is - desired. For example, when using the Opus codec, the "usedtx=1" - parameter would be specified in the offer. This option allows the - endpoint to significantly reduce the amount of audio bandwidth it - receives, at the cost of some fidelity, depending on the quality of - the remote VAD algorithm. + desired. For example, when using the Opus codec [RFC6716], the + "usedtx=1" parameter, specified in [RFC7587], would be used in the + offer. This option allows the endpoint to significantly reduce the + amount of audio bandwidth it receives, at the cost of some fidelity, + depending on the quality of the remote VAD algorithm. If the "VoiceActivityDetection" option is specified, with a value of - "false", the browser MUST NOT emit "CN" codecs. For codecs that have - their own internal silence suppression support, the appropriate fmtp - parameters for that codec MUST be specified to indicate that silence - suppression for received audio is not desired. For example, when - using the Opus codec, the "usedtx=0" parameter would be specified in - the offer. + "false", the JSEP implementation MUST NOT emit "CN" codecs. For + codecs that have their own internal silence suppression support, the + appropriate fmtp parameters for that codec MUST be specified to + indicate that silence suppression for received audio is not desired. + For example, when using the Opus codec, the "usedtx=0" parameter + would be specified in the offer. Note that setting the "VoiceActivityDetection" parameter when generating an offer is a request to receive audio with silence suppression. It has no impact on whether the local endpoint does silence suppression for the audio it sends. The "VoiceActivityDetection" option does not have any impact on the setting of the "vad" value in the signaling of the client to mixer audio level header extension described in [RFC6464], Section 4. @@ -2208,40 +2199,95 @@ Note that the remote description SDP may not have been created by a JSEP endpoint and may not conform to all the requirements listed in Section 5.2. For many cases, this is not a problem. However, if any mandatory SDP attributes are missing, or functionality listed as mandatory-to-use above is not present, this MUST be treated as an error, and MUST cause the affected m= sections to be marked as rejected. The first step in generating an initial answer is to generate session-level attributes. The process here is identical to that - indicated in the Initial Offers section above, except that the + indicated in the initial offers section above, except that the "a=ice-options" line, with the "trickle" option as specified in [I-D.ietf-ice-trickle], Section 4, is only included if such an option was present in the offer. - The next step is to generate session-level lip sync groups as defined - in [RFC5888], Section 7. For each group of type "LS" present in the - offer, determine which of the local RtpTransceivers identified by - that group's mid values reference a common local MediaStream (as - specified in the addTrack and addTransceiver methods). If at least - two such RtpTransceivers exist, a group of type "LS" with the mid - values of these RtpTransceivers MUST be added. Otherwise, this - indicates a difference of opinion between the offerer and answerer - regarding lip sync status, and as such, the offered group MUST be - ignored and no corresponding "LS" group generated. + The next step is to generate session-level lip sync groups, as + defined in [RFC5888], Section 7. For each group of type "LS" present + in the offer, select the local RtpTransceivers that are referenced by + the MID values in the specified group, and determine which of them + either reference a common local MediaStream (specified in the calls + to addTrack/addTransceiver used to create them), or have no + MediaStream to reference because they were not created by addTrack/ + addTransceiver. If at least two such RtpTransceivers exist, a group + of type "LS" with the mid values of these RtpTransceivers MUST be + added. Otherwise the offered "LS" group MUST be ignored and no + corresponding group generated in the answer. + + As a simple example, consider the following offer of a single audio + and single video track contained in the same MediaStream. SDP lines + not relevant to this example have been removed for clarity. As + explained in Section 5.2, a group of type "LS" has been added that + references each track's RtpTransceiver. + + a=group:LS a1 v1 + m=audio 10000 UDP/TLS/RTP/SAVPF 0 + a=mid:a1 + a=msid:ms1 mst1a + m=video 10001 UDP/TLS/RTP/SAVPF 96 + a=mid:v1 + a=msid:ms1 mst1v + + If the answerer uses a single MediaStream when it adds its tracks, + both of its transceivers will reference this stream, and so the + subsequent answer will contain a "LS" group identical to that in the + offer, as shown below: + + a=group:LS a1 v1 + m=audio 20000 UDP/TLS/RTP/SAVPF 0 + a=mid:a1 + a=msid:ms2 mst2a + m=video 20001 UDP/TLS/RTP/SAVPF 96 + a=mid:v1 + a=msid:ms2 mst2v + + However, if the answerer groups its tracks into separate + MediaStreams, its transceivers will reference different streams, and + so the subsequent answer will not contain a "LS" group. + + m=audio 20000 UDP/TLS/RTP/SAVPF 0 + a=mid:a1 + a=msid:ms2a mst2a + m=video 20001 UDP/TLS/RTP/SAVPF 96 + a=mid:v1 + a=msid:ms2b mst2v + + Finally, if the answerer does not add any tracks, its transceivers + will not reference any MediaStreams, causing the preferences of the + offerer to be maintained, and so the subsequent answer will contain + an identical "LS" group. + + a=group:LS a1 v1 + m=audio 20000 UDP/TLS/RTP/SAVPF 0 + a=mid:a1 + a=recvonly + m=video 20001 UDP/TLS/RTP/SAVPF 96 + a=mid:v1 + a=recvonly + + The Section 7.2 example later in this document shows a more involved + case of "LS" group generation. The next step is to generate m= sections for each m= section that is present in the remote offer, as specified in [RFC3264], Section 6. For the purposes of this discussion, any session-level attributes in - the offer that are also valid as media-level attributes SHALL be + the offer that are also valid as media-level attributes are considered to be present in each m= section. The next step is to go through each offered m= section. Each offered m= section will have an associated RtpTransceiver, as described in Section 5.9. If there are more RtpTransceivers than there are m= sections, the unmatched RtpTransceivers will need to be associated in a subsequent offer. For each offered m= section, if any of the following conditions are true, the corresponding m= section in the answer MUST be marked as @@ -2271,21 +2317,21 @@ o The field MUST be set to exactly match the field for the corresponding m= line in the offer. o If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, and MUST exclude any codecs not present in the codec preferences or not present in the offer. Note that non-JSEP endpoints are not subject to this restriction, and might add media formats in the answer that are not present in the offer, as specified in - [RFC3264], Section 6.1. Therefore, JSEP endpoints MUST be + [RFC3264], Section 6.1. Therefore, JSEP implementations MUST be prepared to receive such answers. o Unless excluded by the above restrictions, the media formats MUST include the mandatory audio/video codecs as specified in [I-D.ietf-rtcweb-audio](see Section 3) and [I-D.ietf-rtcweb-video](see Section 5). The m= line MUST be followed immediately by a "c=" line, as specified in [RFC4566], Section 5.7. Again, as no candidates are available yet, the "c=" line must contain the "dummy" value "IN IP4 0.0.0.0", @@ -2305,57 +2351,68 @@ the offered direction specified in [RFC3264], Section 6.1, and then intersecting with the direction of the associated RtpTransceiver. For example, in the case where an m= section is offered as "sendonly", and the local transceiver is set to "sendrecv", the result in the answer is a "recvonly" direction. o For each media format on the m= line, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6, and [RFC3264], Section 6.1. - o If this m= section is for media with configurable durations of - media per packet, e.g., audio, an "a=maxptime" line, as described - in Section 5.2. - - o If this m= section is for video media, and there are known - limitations on the size of images which can be decoded, an - "a=imageattr" line, as specified in Section 3.6. - o If "rtx" is present in the offer, for each primary codec where RTP retransmission should be used, a corresponding "a=rtpmap" line indicating "rtx" with the clock rate of the primary codec and an "a=fmtp" line that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. o For each supported FEC mechanism, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are specified in [I-D.ietf-rtcweb-fec], Section 6, and specific usage for each media type is outlined in Sections 4 and 5. + o If this m= section is for media with configurable durations of + media per packet, e.g., audio, an "a=maxptime" line, as described + in Section 5.2. + + o If this m= section is for video media, and there are known + limitations on the size of images which can be decoded, an + "a=imageattr" line, as specified in Section 3.6. + o For each supported RTP header extension that is present in the offer, an "a=extmap" line, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/MUST be supported is specified in [I-D.ietf-rtcweb-rtp-usage], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4. o For each supported RTCP feedback mechanism that is present in the offer, an "a=rtcp-fb" mechanism, as specified in [RFC4585], Section 4.2. The list of RTCP feedback mechanisms that SHOULD/ MUST be supported is specified in [I-D.ietf-rtcweb-rtp-usage], Section 5.1. o If the RtpTransceiver has a sendrecv or sendonly direction: - * An "a=msid" line, as specified in [I-D.ietf-mmusic-msid], - Section 2. + * For each MediaStream that was associated with the transceiver + when it was created via addTrack or addTransceiver, an "a=msid" + line, as specified in [I-D.ietf-mmusic-msid], Section 2. If a + MediaStreamTrack is attached to the transceiver's RtpSender, + the "a=msid" lines MUST use that track's ID. If no + MediaStreamTrack is attached, a valid ID MUST be generated, in + the same way that the implementation generates IDs for local + tracks. + + * If no MediaStream is associated with the transceiver, a single + "a=msid" line with the special value "-" in place of the + MediaStream ID, as specified in [I-D.ietf-mmusic-msid], + Section 3. The track ID MUST be selected as described above. Each m= section which is not bundled into another m= section, MUST contain the following attributes (which are of category IDENTICAL or TRANSPORT): o "a=ice-ufrag" and "a=ice-pwd" lines, as specified in [RFC5245], Section 15.4. o An "a=fingerprint" line for each of the endpoint's certificates, as specified in [RFC4572], Section 5; the digest algorithm used @@ -2373,29 +2430,47 @@ o If present in the offer, an "a=rtcp-mux" line, as specified in [RFC5761], Section 5.1.3. Otherwise, an "a=rtcp" line, as specified in [RFC3605], Section 2.1, containing the dummy value "9 IN IP4 0.0.0.0" (because no candidates have yet been gathered). o If present in the offer, an "a=rtcp-rsize" line, as specified in [RFC5506], Section 5. If a data channel m= section has been offered, a m= section MUST also be generated for data. The field MUST be set to - "application" and the and "fmt" fields MUST be set to exactly + "application" and the and fields MUST be set to exactly match the fields in the offer. - Within the data m= section, the "a=mid", "a=ice-ufrag", "a=ice-pwd", - "a=candidate", "a=fingerprint", "a=dtls-id", and "a=setup" lines MUST - be included under the conditions described above, along with an - "a=fmtp:webrtc-datachannel" line and an "a=sctp-port" line - referencing the SCTP port number as defined in - [I-D.ietf-mmusic-sctp-sdp], Section 4.1. + Within the data m= section, an "a=mid" line MUST be generated and + included as described above, along with an "a=sctp-port" line + referencing the SCTP port number, as defined in + [I-D.ietf-mmusic-sctp-sdp], Section 5.1, and, if appropriate, an + "a=max-message-size" line, as defined in [I-D.ietf-mmusic-sctp-sdp], + Section 6.1. + + As discussed above, the following attributes of category IDENTICAL or + TRANSPORT are included only if the data m= section is not bundled + into another m= section: + + o "a=ice-ufrag" + + o "a=ice-pwd" + o "a=fingerprint" + + o "a=setup" + + o "a=dtls-id" + + Note that if media m= sections are bundled into a data m= section, + then certain TRANSPORT and IDENTICAL attributes may also appear in + the data m= section even if they would otherwise only be appropriate + for a media m= section (e.g., "a=rtcp-mux"). If "a=group" attributes with semantics of "BUNDLE" are offered, corresponding session-level "a=group" attributes MUST be added as specified in [RFC5888]. These attributes MUST have semantics "BUNDLE", and MUST include the all mid identifiers from the offered bundle groups that have not been rejected. Note that regardless of the presence of "a=bundle-only" in the offer, no m= sections in the answer should have an "a=bundle-only" line. Attributes that are common between all m= sections MAY be moved to @@ -2426,22 +2501,23 @@ o The "s=" and "t=" lines MUST stay the same. o Each "m=" and c=" line MUST be filled in with the port and address of the default candidate for the m= section, as described in [RFC5245], Section 4.3. Note, however, that the m= line protocol need not match the default candidate, because this protocol value must instead match what was supplied in the offer, as described above. - o The media formats on the m= line MUST be generated in the same - order as in the current local description. + o Unless codec preferences have been set for the associated + transceiver, the media formats on the m= line MUST be generated in + the same order as in the current local description. o Each "a=ice-ufrag" and "a=ice-pwd" line MUST stay the same, unless the m= section is restarting, in which case new ICE credentials must be created as specified in [RFC5245], Section 9.2.1.1. If the m= section is bundled into another m= section, it still MUST NOT contain any ICE credentials. o Each "a=setup" line MUST use an "active" or "passive" role value consistent with the existing DTLS association, if the association is being continued by the offerer. @@ -2455,22 +2531,25 @@ o If the m= section is not bundled into another m= section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an "a=candidate" line MUST be added, as defined in [RFC5245], Section 4.3., paragraph 3. If candidate gathering for the section has completed, an "a=end-of-candidates" attribute MUST be added, as described in [I-D.ietf-ice-trickle], Section 9.3. If the m= section is bundled into another m= section, both "a=candidate" and "a=end-of-candidates" MUST be omitted. - o For RtpTransceivers that are not stopped, the "a=msid" line MUST - stay the same. + o For RtpTransceivers that are not stopped, the "a=msid" line(s) + MUST stay the same, regardless of changes to the transceiver's + direction or track. If no "a=msid" line is present in the current + description, "a=msid" line(s) MUST be generated according to the + same rules as for an initial answer. 5.3.3. Options Handling The createAnswer method takes as a parameter an RTCAnswerOptions object. The set of parameters for RTCAnswerOptions is different than those supported in RTCOfferOptions; the IceRestart option is unnecessary, as ICE credentials will automatically be changed for all m= sections where the offerer chose to perform ICE restart. The following options are supported in RTCAnswerOptions. @@ -2501,44 +2580,50 @@ and specifies a subset of what was in the original offer. This is safe because the answer is not permitted to expand capabilities, and therefore will just respond to what is present in the offer. The application SHOULD NOT modify the SDP in the answer it transmits, as the answer contains the negotiated capabilities, and this can cause the two sides to have different ideas about what exactly was negotiated. As always, the application is solely responsible for what it sends to - the other party, and all incoming SDP will be processed by the - browser to the extent of its capabilities. It is an error to assume - that all SDP is well-formed; however, one should be able to assume - that any implementation of this specification will be able to + the other party, and all incoming SDP will be processed by the JSEP + implementation to the extent of its capabilities. It is an error to + assume that all SDP is well-formed; however, one should be able to + assume that any implementation of this specification will be able to process, as a remote offer or answer, unmodified SDP coming from any other implementation of this specification. 5.5. Processing a Local Description When a SessionDescription is supplied to setLocalDescription, the following steps MUST be performed: o First, the type of the SessionDescription is checked against the current state of the PeerConnection: * If the type is "offer", the PeerConnection state MUST be either "stable" or "have-local-offer". * If the type is "pranswer" or "answer", the PeerConnection state MUST be either "have-remote-offer" or "have-local-pranswer". o If the type is not correct for the current state, processing MUST stop and an error MUST be returned. + o The SessionDescription is then checked to ensure that its contents + are identical to those generated in the last call to createOffer/ + createAnswer, and thus have not been altered, as discussed in + Section 5.4; otherwise, processing MUST stop and an error MUST be + returned. + o Next, the SessionDescription is parsed into a data structure, as described in the Section 5.7 section below. If parsing fails for any reason, processing MUST stop and an error MUST be returned. o Finally, the parsed SessionDescription is applied as described in the Section 5.8 section below. 5.6. Processing a Remote Description When a SessionDescription is supplied to setRemoteDescription, the @@ -2644,29 +2729,36 @@ Section 5, and the set of fingerprint and algorithm values is stored. o If present, a single "a=setup" line is parsed as specified in [RFC4145], Section 4, and the setup value is stored. o If present, a single "a=dtls-id" line is parsed as specified in [I-D.ietf-mmusic-dtls-sdp] Section 5, and the dtls-id value is stored. + o Any "a=identity" lines are parsed and the identity values stored + for subsequent verification, as specified + [I-D.ietf-rtcweb-security-arch], Section 5. + o Any "a=extmap" lines are parsed as specified in [RFC5285], Section 5, and their values are stored. + As required by [RFC4566], Section 5.13, unknown attribute lines MUST + be ignored. + Once all the session-level lines have been parsed, processing - continues with the lines in media sections. + continues with the lines in m= sections. 5.7.2. Media Section Parsing - Like the session-level lines, the media session lines MUST occur in + Like the session-level lines, the media section lines MUST occur in the specific order and with the specific syntax defined in [RFC4566], Section 5. The "m=" line itself MUST be parsed as described in [RFC4566], Section 5.14, and the media, port, proto, and fmt values stored. Following the "m=" line, specific processing MUST be applied for the following non-attribute lines: o As with the "c=" line at the session level, the "c=" line MUST be @@ -2698,21 +2790,21 @@ o If present, a single "a=end-of-candidates" attribute MUST be parsed as specified in [I-D.ietf-ice-trickle], Section 8.2, and its presence or absence flagged and stored. o Any "a=fingerprint" lines are parsed as specified in [RFC4572], Section 5, and the set of fingerprint and algorithm values is stored. If the "m=" proto value indicates use of RTP, as described in the - Section 5.1.3 section above, the following attribute lines MUST be + Section 5.1.2 section above, the following attribute lines MUST be processed: o The "m=" fmt value MUST be parsed as specified in [RFC4566], Section 5.14, and the individual values stored. o Any "a=rtpmap" or "a=fmtp" lines MUST be parsed as specified in [RFC4566], Section 6, and their values stored. o If present, a single "a=ptime" line MUST be parsed as described in [RFC4566], Section 6, and its value stored. @@ -2742,23 +2834,22 @@ presence or absence flagged and stored. o If present, a single "a=rtcp-rsize" attribute MUST be parsed as specified in [RFC5506], Section 5, and its presence or absence flagged and stored. o If present, a single "a=rtcp" attribute MUST be parsed as specified in [RFC3605], Section 2.1, but its value is ignored, as this information is superfluous when using ICE. - o If present, a single "a=msid" attribute MUST be parsed as - specified in [I-D.ietf-mmusic-msid], Section 3.2, and its value - stored. + o If present, "a=msid" attributes MUST be parsed as specified in + [I-D.ietf-mmusic-msid], Section 3.2, and their values stored. o Any "a=imageattr" attributes MUST be parsed as specified in [RFC6236], Section 3, and their values stored. o Any "a=rid" lines MUST be parsed as specified in [I-D.ietf-mmusic-rid], Section 10, and their values stored. o If present, a single "a=simulcast" line MUST be parsed as specified in [I-D.ietf-mmusic-sdp-simulcast], and its values stored. @@ -2771,29 +2862,32 @@ protocol value stored. o An "a=sctp-port" attribute MUST be present, and it MUST be parsed as specified in [I-D.ietf-mmusic-sctp-sdp], Section 5.2, and the value stored. o If present, a single "a=max-message-size" attribute MUST be parsed as specified in [I-D.ietf-mmusic-sctp-sdp], Section 6, and the value stored. Otherwise, use the specified default. + As required by [RFC4566], Section 5.13, unknown attribute lines MUST + be ignored. + 5.7.3. Semantics Verification Assuming parsing completes successfully, the parsed description is then evaluated to ensure internal consistency as well as proper support for mandatory features. Specifically, the following checks are performed: o For each m= section, valid values for each of the mandatory-to-use - features enumerated in Section 5.1.2 MUST be present. These + features enumerated in Section 5.1.1 MUST be present. These values MAY either be present at the media level, or inherited from the session level. * ICE ufrag and password values, which MUST comply with the size limits specified in [RFC5245], Section 15.4. * dtls-id value, which MUST be set according to [I-D.ietf-mmusic-dtls-sdp] Section 5. If this is a re-offer and the dtls-id value is different from that presently in use, the DTLS connection is not being continued and the remote @@ -2810,65 +2904,62 @@ present. o All RID values referenced in an "a=simulcast" line MUST exist as "a=rid" lines. o Each m= section is also checked to ensure prohibited features are not used. If this is a local description, the "ice-lite" attribute MUST NOT be specified. o If the RTP/RTCP multiplexing policy is "require", each m= section - MUST contain an "a=rtcp-mux" attribute. + MUST contain an "a=rtcp-mux" attribute. If an "m=" section + contains an "a=rtcp-mux-only" attribute then that section MUST + also contain an "a=rtcp-mux" attribute. If this session description is of type "pranswer" or "answer", the following additional checks are applied: o The session description must follow the rules defined in [RFC3264], Section 6, including the requirement that the number of m= sections MUST exactly match the number of m= sections in the associated offer. o For each m= section, the media type and protocol values MUST exactly match the media type and protocol values in the corresponding m= section in the associated offer. If any of the preceding checks failed, processing MUST stop and an error MUST be returned. 5.8. Applying a Local Description The following steps are performed at the media engine level to apply - a local description. + a local description. If an error is returned, the session MUST be + restored to the state it was in before performing these steps. - First, the parsed parameters are checked to ensure that they are - identical to those generated in the last call to createOffer/ - createAnswer, and thus have not been altered, as discussed in - Section 5.4; otherwise, processing MUST stop and an error MUST be + Next, m= sections are processed. For each m= section, the following + steps MUST be performed; if any parameters are out of bounds, or + cannot be applied, processing MUST stop and an error MUST be returned. - Next, media sections are processed. For each media section, the - following steps MUST be performed; if any parameters are out of - bounds, or cannot be applied, processing MUST stop and an error MUST - be returned. - - o If this media section is new, begin gathering candidates for it, - as defined in [RFC5245], Section 4.1.1, unless it has been marked - as bundle-only. + o If this m= section is new, begin gathering candidates for it, as + defined in [RFC5245], Section 4.1.1, unless it has been marked as + bundle-only. o Or, if the ICE ufrag and password values have changed, and it has - not been marked as bundle-only, trigger the ICE Agent to start an - ICE restart, and begin gathering new candidates for the media - section as described in [RFC5245], Section 9.1.1.1. If this - description is an answer, also start checks on that media section - as defined in [RFC5245], Section 9.3.1.1. + not been marked as bundle-only, trigger the ICE agent to start an + ICE restart, and begin gathering new candidates for the m= section + as described in [RFC5245], Section 9.1.1.1. If this description + is an answer, also start checks on that media section as defined + in [RFC5245], Section 9.3.1.1. - o If the media section proto value indicates use of RTP: + o If the m= section proto value indicates use of RTP: * If there is no RtpTransceiver associated with this m= section (which will only happen when applying an offer), find one and associate it with this m= section according to the following steps: + Find the RtpTransceiver that corresponds to this m= section, using the mapping between transceivers and m= section indices established when creating the offer. @@ -2880,21 +2971,21 @@ Section 5.1.3. If RTCP mux is not indicated, but was previously negotiated, i.e., the RTCP ICE component no longer exists, this MUST result in an error. * For each specified RTP header extension, establish a mapping between the extension ID and URI, as described in section 6 of [RFC5285]. If any indicated RTP header extension is not supported, this MUST result in an error. * If the MID header extension is supported, prepare to demux RTP - streams intended for this media section based on the MID header + streams intended for this m= section based on the MID header extension, as described in [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 14. * For each specified media format, establish a mapping between the payload type and the actual media format, as described in [RFC3264], Section 6.1. If any indicated media format is not supported, this MUST result in an error. * For each specified "rtx" media format, establish a mapping between the RTX payload type and its associated primary payload @@ -2903,71 +2994,81 @@ result in an error. * If the directional attribute is of type "sendrecv" or "recvonly", enable receipt and decoding of media. Finally, if this description is of type "pranswer" or "answer", follow the processing defined in the Section 5.10 section below. 5.9. Applying a Remote Description + The following steps are performed to apply a remote description. If + an error is returned, the session MUST be restored to the state it + was in before performing these steps. + If the answer contains any "a=ice-options" attributes where "trickle" is listed as an attribute, update the PeerConnection canTrickle property to be true. Otherwise, set this property to false. - The following steps are performed at the media engine level to apply - a remote description. - The following steps MUST be performed for attributes at the session level; if any parameters are out of bounds, or cannot be applied, processing MUST stop and an error MUST be returned. o For any specified "CT" bandwidth value, set this as the limit for the maximum total bitrate for all m= sections, as specified in Section 5.8 of [RFC4566]. Within this overall limit, the implementation can dynamically decide how to best allocate the available bandwidth between m= sections, respecting any specific limits that have been specified for individual m= sections. o For any specified "RR" or "RS" bandwidth values, handle as specified in [RFC3556], Section 2. o Any "AS" bandwidth value MUST be ignored, as the meaning of this construct at the session level is not well defined. - For each media section, the following steps MUST be performed; if any + For each m= section, the following steps MUST be performed; if any parameters are out of bounds, or cannot be applied, processing MUST stop and an error MUST be returned. - o If the ICE ufrag or password changed from the previous remote - description, then an ICE restart is needed, as described in - Section 9.1.1.1 of [RFC5245] If the description is of type - "offer", mark that an ICE restart is needed. If the description - is of type "answer" and the current local description is also an - ICE restart, then signal the ICE agent to begin checks as - described in Section 9.3.1.1 of [RFC5245]. An answer MUST change - the ufrag and password in an answer if and only if ICE is - restarting, as described in Section 9.2.1.1 of [RFC5245]. + o If the PeerConnection state is "have-local-offer", and the ICE + ufrag or password changed from the previous remote description, + then an ICE restart is needed, as described in Section 9.1.1.1 of + [RFC5245]. If the description is of type "offer", note that an + ICE restart is needed. If the description is of type "answer" or + "pranswer" and the current local description is also an ICE + restart, then signal the ICE agent to begin checks as described in + Section 9.3.1.1 of [RFC5245]. An answerer MUST change the ufrag + and password in an answer if and only if ICE is restarting, as + described in Section 9.2.1.1 of [RFC5245]. + + o If the PeerConnection state is "have-remote-pranswer", and the ICE + ufrag or password changed from the previous provisional answer, + then signal the ICE agent to discard any previous ICE check list + state for the m= section and begin checks as if this were the + first answer. However, such an answer MAY only change the ICE + ufrag or password if the local offer is starting or restarting ICE + for the m= section. o Configure the ICE components associated with this media section to use the supplied ICE remote ufrag and password for their connectivity checks. o Pair any supplied ICE candidates with any gathered local candidates, as described in Section 5.7 of [RFC5245] and start connectivity checks with the appropriate credentials. o If an "a=end-of-candidates" attribute is present, process the end- of-candidates indication as described in [I-D.ietf-ice-trickle] Section 11. - o If the media section proto value indicates use of RTP: + o If the m= section proto value indicates use of RTP: * If the m= section is being recycled (see Section 5.2.2), dissociate the currently associated RtpTransceiver by setting its mid property to null, and discard the mapping between the transceiver and its m= section index. * If the m= section is not associated with any RtpTransceiver (possibly because it was dissociated in the previous step), either find an RtpTransceiver or create one according to the following steps: @@ -3039,21 +3140,21 @@ 5% to RTCP. "TIAS" is used in preference to "AS" because it provides more accurate control of bandwidth. * For any "RR" or "RS" bandwidth values, handle as specified in [RFC3556], Section 2. * Any specified "CT" bandwidth value MUST be ignored, as the meaning of this construct at the media level is not well defined. - * If the media section is of type audio: + * If the m= section is of type audio: + For each specified "CN" media format, enable DTX for all supported media formats with the same clockrate, as described in [RFC3389], Section 5, except for formats that have their own internal DTX mechanisms. DTX for such formats (e.g., Opus) is controlled via fmtp parameters, as discussed in Section 5.2.3.2. + For each specified "telephone-event" media format, enable DTMF transmission for all supported media formats with the @@ -3069,65 +3170,71 @@ Finally, if this description is of type "pranswer" or "answer", follow the processing defined in the Section 5.10 section below. 5.10. Applying an Answer In addition to the steps mentioned above for processing a local or remote description, the following steps are performed when processing a description of type "pranswer" or "answer". - For each media section, the following steps MUST be performed: + For each m= section, the following steps MUST be performed: - o If the media section has been rejected (i.e. port is set to zero - in the answer), stop any reception or transmission of media for - this section, and, unless a non-rejected media section is bundled - with this media section, discard any associated ICE components, as + o If the m= section has been rejected (i.e. port is set to zero in + the answer), stop any reception or transmission of media for this + section, and, unless a non-rejected m= section is bundled with + this m= section, discard any associated ICE components, as described in Section 9.2.1.3 of [RFC5245]. o If the remote DTLS fingerprint has been changed or the dtls-id has - changed, tear down the DTLS connection. If a DTLS connection - needs to be torn down but the answer does not indicate an ICE - restart, an error MUST be generated. If an ICE restart is - performed without a change in dtls-id or fingerprint, then the - same DTLS connection is continued over the new ICE channel. + changed, tear down the DTLS connection. This includes the case + when the PeerConnection state is "have-remote-pranswer". If a + DTLS connection needs to be torn down but the answer does not + indicate an ICE restart or, in the case of "have-remote-pranswer", + new ICE credentials, an error MUST be generated. If an ICE + restart is performed without a change in dtls-id or fingerprint, + then the same DTLS connection is continued over the new ICE + channel. o If no valid DTLS connection exists, prepare to start a DTLS connection, using the specified roles and fingerprints, on any underlying ICE components, once they are active. - o If the media section proto value indicates use of RTP: + o If the m= section proto value indicates use of RTP: - * If the media section references any media formats, RTP header + * If the m= section references any media formats, RTP header extensions, or RTCP feedback mechanisms that were not present - in the corresponding media section in the offer, this indicates - a negotiation problem and MUST result in an error. + in the corresponding m= section in the offer, this indicates a + negotiation problem and MUST result in an error. - * If the media section has RTCP mux enabled, discard the RTCP ICE + * If the m= section has RTCP mux enabled, discard the RTCP ICE component, if one exists, and begin or continue muxing RTCP over the RTP ICE component, as specified in [RFC5761], Section 5.1.3. Otherwise, prepare to transmit RTCP over the RTCP ICE component; if no RTCP ICE component exists, because RTCP mux was previously enabled, this MUST result in an error. - * If the media section has reduced-size RTCP enabled, configure - the RTCP transmission for this media section to use reduced- - size RTCP, as specified in [RFC5506]. + * If the m= section has reduced-size RTCP enabled, configure the + RTCP transmission for this m= section to use reduced-size RTCP, + as specified in [RFC5506]. * If the directional attribute in the answer is of type "sendrecv" or "sendonly", choose the media format to send as the most preferred media format from the remote description that is also present in the answer, as described in [RFC3264], Sections 6.1 and 7, and start transmitting RTP media once the - underlying transport layers have been established. If a SSRC + underlying transport layers have been established. If an SSRC has not already been chosen for this outgoing RTP stream, - choose a random one. + choose a random one. If media is already being transmitted, + the same SSRC SHOULD be used unless the clockrate of the new + codec is different, in which case a new SSRC MUST be chosen, as + specified in [RFC7160], Section 3.1. * The payload type mapping from the remote description is used to determine payload types for the outgoing RTP streams, including the payload type for the send media format chosen above. Any RTP header extensions that were negotiated should be included in the outgoing RTP streams, using the extension mapping from the remote description; if the RID header extension has been negotiated, and RID values are specified, include the RID header extension in the outgoing RTP streams, as indicated in [I-D.ietf-mmusic-rid], Section 4. @@ -3156,23 +3263,28 @@ feedback types and reacting to received feedback, as specified in [RFC4585], Section 4.2. When sending RTCP feedback, follow the rules and recommendations from [I-D.ietf-avtcore-rtp-multi-stream], Section 5.4.1 to select which SSRC to use. * If the directional attribute is of type "recvonly" or "inactive", stop transmitting all RTP media, but continue sending RTCP, as described in [RFC3264], Section 5.1. - o If the media section proto value indicates use of SCTP: + o If the m= section proto value indicates use of SCTP: - * If no SCTP association yet exists, prepare to initiate a SCTP + * If an SCTP association exists, and the remote SCTP port has + changed, discard the existing SCTP association. This includes + the case when the PeerConnection state is "have-remote- + pranswer". + + * If no valid SCTP association exists, prepare to initiate a SCTP association over the associated ICE component and DTLS connection, using the local SCTP port value from the local description, and the remote SCTP port value from the remote description, as described in [I-D.ietf-mmusic-sctp-sdp], Section 10.2. If the answer contains valid bundle groups, discard any ICE components for the m= sections that will be bundled onto the primary ICE components in each bundle, and begin muxing these m= sections accordingly, as described in @@ -3204,616 +3316,935 @@ multiple lines, where leading whitespace indicates that a line is a continuation of the previous line. In addition, some blank lines have been added to improve readability but are not valid in SDP. More examples of SDP for WebRTC call flows can be found in [I-D.nandakumar-rtcweb-sdp]. 7.1. Simple Example This section shows a very simple example that sets up a minimal audio - / video call between two browsers and does not use trickle ICE. The - example in the following section provides a more realistic example of - what would happen in a normal browser to browser connection. + / video call between two JSEP endpoints without using trickle ICE. + The example in the following section provides a more detailed example + of what could happen in a JSEP session. - The flow shows Alice's browser initiating the session to Bob's - browser. The messages from Alice's JS to Bob's JS are assumed to - flow over some signaling protocol via a web server. The JS on both - Alice's side and Bob's side waits for all candidates before sending - the offer or answer, so the offers and answers are complete. Trickle - ICE is not used. Both Alice and Bob are using the default policy of - balanced. + The code flow below shows Alice's endpoint initiating the session to + Bob's endpoint. The messages from Alice's JS to Bob's JS are assumed + to flow over some signaling protocol via a web server. The JS on + both Alice's side and Bob's side waits for all candidates before + sending the offer or answer, so the offers and answers are complete; + trickle ICE is not used. Both Alice and Bob are using the default + bundle policy of "balanced", and the default RTCP mux policy of + "require". // set up local media state AliceJS->AliceUA: create new PeerConnection AliceJS->AliceUA: addTrack with two tracks: audio and video AliceJS->AliceUA: createOffer to get offer AliceJS->AliceUA: setLocalDescription with offer AliceUA->AliceJS: multiple onicecandidate events with candidates // wait for ICE gathering to complete AliceUA->AliceJS: onicecandidate event with null candidate AliceJS->AliceUA: get |offer-A1| from pendingLocalDescription // |offer-A1| is sent over signaling protocol to Bob AliceJS->WebServer: signaling with |offer-A1| WebServer->BobJS: signaling with |offer-A1| // |offer-A1| arrives at Bob BobJS->BobUA: create a PeerConnection BobJS->BobUA: setRemoteDescription with |offer-A1| -BobUA->BobJS: onaddstream event with remoteStream +BobUA->BobJS: ontrack events for audio and video tracks // Bob accepts call BobJS->BobUA: addTrack with local tracks BobJS->BobUA: createAnswer BobJS->BobUA: setLocalDescription with answer BobUA->BobJS: multiple onicecandidate events with candidates // wait for ICE gathering to complete BobUA->BobJS: onicecandidate event with null candidate BobJS->BobUA: get |answer-A1| from currentLocalDescription // |answer-A1| is sent over signaling protocol to Alice BobJS->WebServer: signaling with |answer-A1| WebServer->AliceJS: signaling with |answer-A1| // |answer-A1| arrives at Alice AliceJS->AliceUA: setRemoteDescription with |answer-A1| -AliceUA->AliceJS: onaddstream event with remoteStream +AliceUA->AliceJS: ontrack events for audio and video tracks // media flows BobUA->AliceUA: media sent from Bob to Alice AliceUA->BobUA: media sent from Alice to Bob The SDP for |offer-A1| looks like: v=0 o=- 4962303333179871722 1 IN IP4 0.0.0.0 s=- t=0 0 - a=group:BUNDLE a1 v1 a=ice-options:trickle - m=audio 56500 UDP/TLS/RTP/SAVPF 96 0 8 97 98 - c=IN IP4 192.0.2.1 + a=group:BUNDLE a1 v1 + a=group:LS a1 v1 + + m=audio 10100 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 203.0.113.100 a=mid:a1 - a=rtcp:56501 IN IP4 192.0.2.1 - a=msid:47017fee-b6c1-4162-929c-a25110252400 - f83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:ETEn1v9DoTMB9J4r + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:47017fee-b6c1-4162-929c-a25110252400 + f83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 + a=ice-ufrag:ETEn a=ice-pwd:OtSK0WpNtpUjkY4+86js7ZQl a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 + 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04: + BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 a=setup:actpass + a=dtls-id:1 + a=rtcp:10101 IN IP4 203.0.113.100 a=rtcp-mux a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid - a=candidate:3348148302 1 udp 2113937151 192.0.2.1 56500 - typ host - a=candidate:3348148302 2 udp 2113937151 192.0.2.1 56501 - typ host + a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host + a=candidate:1 2 udp 2113929470 203.0.113.100 10101 typ host a=end-of-candidates - m=video 56502 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 192.0.2.1 - a=rtcp:56503 IN IP4 192.0.2.1 + m=video 10102 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 203.0.113.100 a=mid:v1 - a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae - f30bdb4a-5db8-49b5-bcdc-e0c9a23172e0 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 - a=ice-ufrag:BGKkWnG5GmiUpdIV + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtcp-fb:100 ccm fir + a=rtcp-fb:100 nack + a=rtcp-fb:100 nack pli + a=msid:47017fee-b6c1-4162-929c-a25110252400 + f30bdb4a-5db8-49b5-bcdc-e0c9a23172e0 + a=ice-ufrag:BGKk a=ice-pwd:mqyWsAjvtKwTGnvhPztQ9mIf a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - + 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04: + BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 a=setup:actpass + a=dtls-id:1 + a=rtcp:10103 IN IP4 203.0.113.100 a=rtcp-mux a=rtcp-rsize - a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid - a=rtcp-fb:100 ccm fir - a=rtcp-fb:100 nack - a=rtcp-fb:100 nack pli - a=candidate:3348148302 1 udp 2113937151 192.0.2.1 56502 - typ host - a=candidate:3348148302 2 udp 2113937151 192.0.2.1 56503 - typ host + a=candidate:1 1 udp 2113929471 203.0.113.100 10102 typ host + a=candidate:1 2 udp 2113929470 203.0.113.100 10103 typ host a=end-of-candidates The SDP for |answer-A1| looks like: v=0 o=- 6729291447651054566 1 IN IP4 0.0.0.0 s=- t=0 0 + a=ice-options:trickle a=group:BUNDLE a1 v1 - m=audio 20000 UDP/TLS/RTP/SAVPF 96 0 8 97 98 - c=IN IP4 192.0.2.2 + a=group:LS a1 v1 + + m=audio 10200 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 203.0.113.200 a=mid:a1 - a=rtcp:20000 IN IP4 192.0.2.2 - a=msid:PI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1 - PI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1a0 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:6sFvz2gdLkEwjZEr + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae + 5a7b57b8-f043-4bd1-a45d-09d4dfa31226 + a=ice-ufrag:6sFv a=ice-pwd:cOTZKZNVlO9RSGsEGM63JXT2 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 + a=fingerprint:sha-256 + 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35: + DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 a=setup:active + a=dtls-id:1 a=rtcp-mux a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=candidate:2299743422 1 udp 2113937151 192.0.2.2 20000 - typ host + a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host a=end-of-candidates - m=video 20000 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 192.0.2.2 - a=rtcp 20001 IN IP4 192.0.2.2 + m=video 10200 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 203.0.113.200 a=mid:v1 - a=msid:PI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1 - PI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1v0 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 - a=setup:active - a=rtcp-mux - a=rtcp-rsize + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli + a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae + 4ea4d4a1-2fda-4511-a9cc-1b32c2e59552 -7.2. Normal Examples +7.2. Detailed Example - This section shows a typical example of a session between two - browsers setting up an audio channel and a data channel. Trickle ICE - is used in full trickle mode with a bundle policy of max-bundle, an - RTCP mux policy of require, and a single TURN server. Later, two - video flows, one for the presenter and one for screen sharing, are - added to the session. This example shows Alice's browser initiating - the session to Bob's browser. The messages from Alice's JS to Bob's - JS are assumed to flow over some signaling protocol via a web server. + This section shows a more involved example of a session between two + JSEP endpoints. Trickle ICE is used in full trickle mode, with a + bundle policy of "max-bundle", an RTCP mux policy of "require", and a + single TURN server. Initially, both Alice and Bob establish an audio + channel and a data channel. Later, Bob adds two video flows, one for + his video feed, and one for screensharing, both supporting FEC, and + with the video feed configured for simulcast. Alice accepts these + video flows, but does not add video flows of her own, so they are + handled as recvonly. Alice also specifies a maximum video decoder + resolution. // set up local media state AliceJS->AliceUA: create new PeerConnection AliceJS->AliceUA: addTrack with an audio track AliceJS->AliceUA: createDataChannel to get data channel AliceJS->AliceUA: createOffer to get |offer-B1| AliceJS->AliceUA: setLocalDescription with |offer-B1| // |offer-B1| is sent over signaling protocol to Bob AliceJS->WebServer: signaling with |offer-B1| WebServer->BobJS: signaling with |offer-B1| // |offer-B1| arrives at Bob BobJS->BobUA: create a PeerConnection BobJS->BobUA: setRemoteDescription with |offer-B1| - BobUA->BobJS: onaddstream with audio track from Alice + BobUA->BobJS: ontrack with audio track from Alice // candidates are sent to Bob - AliceUA->AliceJS: onicecandidate event with |candidate-B1| (host) - AliceJS->WebServer: signaling with |candidate-B1| - AliceUA->AliceJS: onicecandidate event with |candidate-B2| (srflx) - AliceJS->WebServer: signaling with |candidate-B2| + AliceUA->AliceJS: onicecandidate (host) |offer-B1-candidate-1| + AliceJS->WebServer: signaling with |offer-B1-candidate-1| + AliceUA->AliceJS: onicecandidate (srflx) |offer-B1-candidate-2| + AliceJS->WebServer: signaling with |offer-B1-candidate-2| + AliceUA->AliceJS: onicecandidate (relay) |offer-B1-candidate-3| + AliceJS->WebServer: signaling with |offer-B1-candidate-3| - WebServer->BobJS: signaling with |candidate-B1| - BobJS->BobUA: addIceCandidate with |candidate-B1| - WebServer->BobJS: signaling with |candidate-B2| - BobJS->BobUA: addIceCandidate with |candidate-B2| + WebServer->BobJS: signaling with |offer-B1-candidate-1| + BobJS->BobUA: addIceCandidate with |offer-B1-candidate-1| + WebServer->BobJS: signaling with |offer-B1-candidate-2| + BobJS->BobUA: addIceCandidate with |offer-B1-candidate-2| + WebServer->BobJS: signaling with |offer-B1-candidate-3| + BobJS->BobUA: addIceCandidate with |offer-B1-candidate-3| // Bob accepts call BobJS->BobUA: addTrack with local audio BobJS->BobUA: createDataChannel to get data channel BobJS->BobUA: createAnswer to get |answer-B1| BobJS->BobUA: setLocalDescription with |answer-B1| // |answer-B1| is sent to Alice BobJS->WebServer: signaling with |answer-B1| WebServer->AliceJS: signaling with |answer-B1| AliceJS->AliceUA: setRemoteDescription with |answer-B1| - AliceUA->AliceJS: onaddstream event with audio track from Bob + AliceUA->AliceJS: ontrack event with audio track from Bob // candidates are sent to Alice - BobUA->BobJS: onicecandidate event with |candidate-B3| (host) - BobJS->WebServer: signaling with |candidate-B3| - BobUA->BobJS: onicecandidate event with |candidate-B4| (srflx) - BobJS->WebServer: signaling with |candidate-B4| + BobUA->BobJS: onicecandidate (host) |answer-B1-candidate-1| + BobJS->WebServer: signaling with |answer-B1-candidate-1| + BobUA->BobJS: onicecandidate (srflx) |answer-B1-candidate-2| + BobJS->WebServer: signaling with |answer-B1-candidate-2| + BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-3| + BobJS->WebServer: signaling with |answer-B1-candidate-3| - WebServer->AliceJS: signaling with |candidate-B3| - AliceJS->AliceUA: addIceCandidate with |candidate-B3| - WebServer->AliceJS: signaling with |candidate-B4| - AliceJS->AliceUA: addIceCandidate with |candidate-B4| + WebServer->AliceJS: signaling with |answer-B1-candidate-1| + AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1| + WebServer->AliceJS: signaling with |answer-B1-candidate-2| + AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-2| + WebServer->AliceJS: signaling with |answer-B1-candidate-3| + AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-3| // data channel opens BobUA->BobJS: ondatachannel event AliceUA->AliceJS: ondatachannel event BobUA->BobJS: onopen AliceUA->AliceJS: onopen - // media is flowing between browsers + // media is flowing between endpoints BobUA->AliceUA: audio+data sent from Bob to Alice AliceUA->BobUA: audio+data sent from Alice to Bob // some time later Bob adds two video streams // note, no candidates exchanged, because of bundle BobJS->BobUA: addTrack with first video stream BobJS->BobUA: addTrack with second video stream BobJS->BobUA: createOffer to get |offer-B2| BobJS->BobUA: setLocalDescription with |offer-B2| // |offer-B2| is sent to Alice BobJS->WebServer: signaling with |offer-B2| WebServer->AliceJS: signaling with |offer-B2| AliceJS->AliceUA: setRemoteDescription with |offer-B2| - AliceUA->AliceJS: onaddstream event with first video stream - AliceUA->AliceJS: onaddstream event with second video stream + AliceUA->AliceJS: ontrack event with first video track + AliceUA->AliceJS: ontrack event with second video track AliceJS->AliceUA: createAnswer to get |answer-B2| AliceJS->AliceUA: setLocalDescription with |answer-B2| // |answer-B2| is sent over signaling protocol to Bob AliceJS->WebServer: signaling with |answer-B2| WebServer->BobJS: signaling with |answer-B2| BobJS->BobUA: setRemoteDescription with |answer-B2| - // media is flowing between browsers + // media is flowing between endpoints BobUA->AliceUA: audio+video+data sent from Bob to Alice AliceUA->BobUA: audio+video+data sent from Alice to Bob The SDP for |offer-B1| looks like: v=0 o=- 4962303333179871723 1 IN IP4 0.0.0.0 s=- t=0 0 - a=group:BUNDLE a1 d1 a=ice-options:trickle + a=group:BUNDLE a1 d1 + m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 0.0.0.0 - a=rtcp:9 IN IP4 0.0.0.0 a=mid:a1 - a=msid:57017fee-b6c1-4162-929c-a25110252400 - e83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:ATEn1v9DoTMB9J4r + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:57017fee-b6c1-4162-929c-a25110252400 + e83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 + a=ice-ufrag:ATEn a=ice-pwd:AtSK0WpNtpUjkY4+86js7ZQl a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 + 29:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04: + BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 a=setup:actpass + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid m=application 0 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 0.0.0.0 - a=bundle-only a=mid:d1 - a=fmtp:webrtc-datachannel max-message-size=65536 - a=sctp-port 5000 - a=fingerprint:sha-256 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - a=setup:actpass + a=sctp-port:5000 + a=max-message-size:65536 + a=bundle-only - The SDP for |candidate-B1| looks like: + |offer-B1-candidate-1| looks like: - candidate:109270923 1 udp 2122194687 192.168.1.2 51556 typ host + ufrag ATEn + index 0 + mid a1 + attr candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host + |offer-B1-candidate-2| looks like: - The SDP for |candidate-B2| looks like: + ufrag ATEn + index 0 + mid a1 + attr candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx + raddr 203.0.113.100 rport 10100 - candidate:4036177503 1 udp 1685987071 11.22.33.44 52546 typ srflx - raddr 192.168.1.2 rport 51556 + |offer-B1-candidate-3| looks like: + + ufrag ATEn + index 0 + mid a1 + attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay + raddr 198.51.100.100 rport 11100 The SDP for |answer-B1| looks like: v=0 o=- 7729291447651054566 1 IN IP4 0.0.0.0 s=- t=0 0 - a=group:BUNDLE a1 d1 a=ice-options:trickle + a=group:BUNDLE a1 d1 + m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 0.0.0.0 - a=rtcp:9 IN IP4 0.0.0.0 a=mid:a1 - a=msid:QI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1 - QI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1a0 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:7sFvz2gdLkEwjZEr + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae + 6a7b57b8-f043-4bd1-a45d-09d4dfa31226 + a=ice-ufrag:7sFv a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 + a=fingerprint:sha-256 + 7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35: + DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 a=setup:active + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid m=application 9 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 0.0.0.0 a=mid:d1 - a=fmtp:webrtc-datachannel max-message-size=65536 - a=sctp-port 5000 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 - a=setup:active + a=sctp-port:5000 + a=max-message-size:65536 - The SDP for |candidate-B3| looks like: + |answer-B1-candidate-1| looks like: - candidate:109270924 1 udp 2122194687 192.168.2.3 61665 typ host + ufrag 7sFv + index 0 + mid a1 + attr candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host + |answer-B1-candidate-2| looks like: - The SDP for |candidate-B4| looks like: + ufrag 7sFv + index 0 + mid a1 + attr candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx + raddr 203.0.113.200 rport 10200 - candidate:4036177504 1 udp 1685987071 55.66.77.88 64532 typ srflx - raddr 192.168.2.3 rport 61665 + |answer-B1-candidate-3| looks like: - The SDP for |offer-B2| looks like: (note the increment of the version - number in the o= line, and the c= and a=rtcp lines, which indicate - the local candidate that was selected) + ufrag 7sFv + index 0 + mid a1 + attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay + raddr 198.51.100.200 rport 11200 + + The SDP for |offer-B2| is shown below. In addition to the new m= + sections for video, both of which are offering FEC, and one of which + is offering simulcast, note the increment of the version number in + the o= line, changes to the c= line, indicating the local candidate + that was selected, and the inclusion of gathered candidates as + a=candidate lines. v=0 o=- 7729291447651054566 2 IN IP4 0.0.0.0 s=- t=0 0 - a=group:BUNDLE a1 d1 v1 v2 a=ice-options:trickle - m=audio 64532 UDP/TLS/RTP/SAVPF 96 0 8 97 98 - c=IN IP4 55.66.77.88 - a=rtcp:64532 IN IP4 55.66.77.88 + a=group:BUNDLE a1 d1 v1 v2 + a=group:LS a1 v1 + + m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 192.0.2.200 a=mid:a1 - a=msid:QI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1 - QI39StLS8W7ZbQl1sJsWUXkr3Zf12fJUvzQ1a0 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:7sFvz2gdLkEwjZEr + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae + 6a7b57b8-f043-4bd1-a45d-09d4dfa31226 + a=ice-ufrag:7sFv a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 + a=fingerprint:sha-256 + 7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35: + DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 a=setup:actpass + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid - a=candidate:109270924 1 udp 2122194687 192.168.2.3 61665 typ host - a=candidate:4036177504 1 udp 1685987071 55.66.77.88 64532 typ srflx - raddr 192.168.2.3 rport 61665 - a=candidate:3671762467 1 udp 41819903 66.77.88.99 50416 typ relay - raddr 55.66.77.88 rport 64532 - + a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host + a=candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx + raddr 203.0.113.200 rport 10200 + a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay + raddr 198.51.100.200 rport 11200 a=end-of-candidates - m=application 64532 UDP/DTLS/SCTP webrtc-datachannel - c=IN IP4 55.66.77.88 + m=application 12200 UDP/DTLS/SCTP webrtc-datachannel + c=IN IP4 192.0.2.200 a=mid:d1 - a=fmtp:webrtc-datachannel max-message-size=65536 - a=sctp-port 5000 - a=ice-ufrag:7sFvz2gdLkEwjZEr - a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2 - a=fingerprint:sha-256 6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35 - :DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08 - a=setup:actpass - a=candidate:109270924 1 udp 2122194687 192.168.2.3 61665 typ host - a=candidate:4036177504 1 udp 1685987071 55.66.77.88 64532 typ srflx - raddr 192.168.2.3 rport 61665 - a=candidate:3671762467 1 udp 41819903 66.77.88.99 50416 typ relay - raddr 55.66.77.88 rport 64532 - a=end-of-candidates + a=sctp-port:5000 + a=max-message-size:65536 - m=video 0 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 55.66.77.88 - a=bundle-only - a=rtcp:64532 IN IP4 55.66.77.88 + m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 + c=IN IP4 192.0.2.200 a=mid:v1 - a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae - f30bdb4a-5db8-49b5-bcdc-e0c9a23172e0 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 - a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - a=setup:actpass - a=rtcp-mux - a=rtcp-rsize - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtpmap:102 flexfec/90000 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli - - m=video 0 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 55.66.77.88 - a=bundle-only - a=rtcp:64532 IN IP4 55.66.77.88 - a=mid:v1 a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae - f30bdb4a-5db8-49b5-bcdc-e0c9a23172e0 + 5ea4d4a1-2fda-4511-a9cc-1b32c2e59552 + a=rid:1 send + a=rid:2 send + a=rid:3 send + a=simulcast:send 1;2;3 + m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 + c=IN IP4 192.0.2.200 + a=mid:v2 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 - a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - a=setup:actpass - a=rtcp-mux - a=rtcp-rsize - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtpmap:102 flexfec/90000 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli + a=msid:81317484-2ed4-49d7-9eb7-1414322a7aae + 6ea4d4a1-2fda-4511-a9cc-1b32c2e59552 - The SDP for |answer-B2| looks like: (note the use of setup:passive to - maintain the existing DTLS roles, and the use of a=recvonly to - indicate that the video streams are one-way) + The SDP for |answer-B2| is shown below. In addition to the + acceptance of the video m= sections, the use of a=recvonly to + indicate one-way video, and the use of a=imageattr to limit the + received resolution, note the use of setup:passive to maintain the + existing DTLS roles. v=0 o=- 4962303333179871723 2 IN IP4 0.0.0.0 s=- t=0 0 - a=group:BUNDLE a1 d1 v1 v2 a=ice-options:trickle - m=audio 52546 UDP/TLS/RTP/SAVPF 96 0 8 97 98 - c=IN IP4 11.22.33.44 - a=rtcp:52546 IN IP4 11.22.33.44 + a=group:BUNDLE a1 d1 v1 v2 + a=group:LS a1 v1 + + m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 192.0.2.100 a=mid:a1 - a=msid:57017fee-b6c1-4162-929c-a25110252400 - e83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=maxptime:120 - a=ice-ufrag:ATEn1v9DoTMB9J4r + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:57017fee-b6c1-4162-929c-a25110252400 + e83006c5-a0ff-4e0a-9ed9-d3e6747be7d9 + a=ice-ufrag:ATEn a=ice-pwd:AtSK0WpNtpUjkY4+86js7ZQl a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 + 29:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04: + BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 a=setup:passive + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid - a=candidate:109270923 1 udp 2122194687 192.168.1.2 51556 typ host - a=candidate:4036177503 1 udp 1685987071 11.22.33.44 52546 typ srflx - raddr 192.168.1.2 rport 51556 - a=candidate:3671762466 1 udp 41819903 22.33.44.55 61405 typ relay - raddr 11.22.33.44 rport 52546 + a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host + a=candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx + raddr 203.0.113.100 rport 10100 + a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay + raddr 198.51.100.100 rport 11100 a=end-of-candidates - m=application 52546 UDP/DTLS/SCTP webrtc-datachannel - c=IN IP4 11.22.33.44 + m=application 12100 UDP/DTLS/SCTP webrtc-datachannel + c=IN IP4 192.0.2.100 a=mid:d1 - a=fmtp:webrtc-datachannel max-message-size=65536 - a=sctp-port 5000 - a=fingerprint:sha-256 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - a=setup:passive + a=sctp-port:5000 + a=max-message-size:65536 - m=video 52546 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 11.22.33.44 - a=rtcp:52546 IN IP4 11.22.33.44 + m=video 12100 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 192.0.2.100 a=mid:v1 a=recvonly a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 + a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0] + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtcp-fb:100 ccm fir + a=rtcp-fb:100 nack + a=rtcp-fb:100 nack pli + + m=video 12100 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 192.0.2.100 + a=mid:v2 + a=recvonly + a=rtpmap:100 VP8/90000 + a=rtpmap:101 rtx/90000 + a=fmtp:101 apt=100 + a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0] + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtcp-fb:100 ccm fir + a=rtcp-fb:100 nack + a=rtcp-fb:100 nack pli + +7.3. Early Transport Warmup Example + + This example demonstrates the early warmup technique described in + Section 4.1.8.1. Here, Alice's endpoint sends an offer to Bob's + endpoint to start an audio/video call. Bob immediately responds with + an answer that accepts the audio/video m= sections, but marks them as + sendonly (from his perspective), meaning that Alice will not yet send + media. This allows the JSEP implementation to start negotiating ICE + and DTLS immediately. Bob's endpoint then prompts him to answer the + call, and when he does, his endpoint sends a second offer which + enables the audio and video m= sections, and thereby bidirectional + media transmission. The advantage of such a flow is that as soon as + the first answer is received, the implementation can proceed with ICE + and DTLS negotiation and establish the session transport. If the + transport setup completes before the second offer is sent, then media + can be transmitted immediately by the callee immediately upon + answering the call, minimizing perceived post-dial-delay. The second + offer/answer exchange can also change the preferred codecs or other + session parameters. + + This example also makes use of the "relay" ICE candidate policy + described in Section 3.5.3 to minimize the ICE gathering and checking + needed. + +// set up local media state +AliceJS->AliceUA: create new PeerConnection with "relay" ICE policy +AliceJS->AliceUA: addTrack with two tracks: audio and video +AliceJS->AliceUA: createOffer to get |offer-C1| +AliceJS->AliceUA: setLocalDescription with |offer-C1| + +// |offer-C1| is sent over signaling protocol to Bob +AliceJS->WebServer: signaling with |offer-C1| +WebServer->BobJS: signaling with |offer-C1| + +// |offer-C1| arrives at Bob +BobJS->BobUA: create new PeerConnection with "relay" ICE policy +BobJS->BobUA: setRemoteDescription with |offer-C1| +BobUA->BobJS: ontrack events for audio and video + +// a relay candidate is sent to Bob +AliceUA->AliceJS: onicecandidate (relay) |offer-C1-candidate-1| +AliceJS->WebServer: signaling with |offer-C1-candidate-1| + +WebServer->BobJS: signaling with |offer-C1-candidate-1| +BobJS->BobUA: addIceCandidate with |offer-C1-candidate-1| + +// Bob prepares an early answer to warm up the transport +BobJS->BobUA: addTransceiver with null audio and video tracks +BobJS->BobUA: transceiver.setDirection(sendonly) for both +BobJS->BobUA: createAnswer +BobJS->BobUA: setLocalDescription with answer + +// |answer-C1| is sent over signaling protocol to Alice +BobJS->WebServer: signaling with |answer-C1| +WebServer->AliceJS: signaling with |answer-C1| + +// |answer-C1| (sendonly) arrives at Alice +AliceJS->AliceUA: setRemoteDescription with |answer-C1| +AliceUA->AliceJS: ontrack events for audio and video + +// a relay candidate is sent to Alice +BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-1| +BobJS->WebServer: signaling with |answer-B1-candidate-1| + +WebServer->AliceJS: signaling with |answer-B1-candidate-1| +AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1| + +// ICE and DTLS establish while call is ringing + +// Bob accepts call, starts media, and sends a new offer +BobJS->BobUA: transceiver.setTrack with audio and video tracks +BobUA->AliceUA: media sent from Bob to Alice +BobJS->BobUA: transceiver.setDirection(sendrecv) for both + transceivers +BobJS->BobUA: createOffer +BobJS->BobUA: setLocalDescription with offer + +// |offer-C2| is sent over signaling protocol to Alice +BobJS->WebServer: signaling with |offer-C2| +WebServer->AliceJS: signaling with |offer-C2| + +// |offer-C2| (sendrecv) arrives at Alice +AliceJS->AliceUA: setRemoteDescription with |offer-C2| +AliceJS->AliceUA: createAnswer +AliceJS->AliceUA: setLocalDescription with |answer-C2| +AliceUA->BobUA: media sent from Alice to Bob + +// |answer-C2| is sent over signaling protocol to Bob +AliceJS->WebServer: signaling with |answer-C2| +WebServer->BobJS: signaling with |answer-C2| +BobJS->BobUA: setRemoteDescription with |answer-C2| + + The SDP for |offer-C1| looks like: + + v=0 + o=- 1070771854436052752 1 IN IP4 0.0.0.0 + s=- + t=0 0 + a=ice-options:trickle + a=group:BUNDLE a1 v1 + a=group:LS a1 v1 + + m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 0.0.0.0 + a=mid:a1 + a=sendrecv + a=rtpmap:96 opus/48000/2 + a=rtpmap:0 PCMU/8000 + a=rtpmap:8 PCMA/8000 + a=rtpmap:97 telephone-event/8000 + a=rtpmap:98 telephone-event/48000 + a=maxptime:120 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce + e80098db-7159-3c06-229a-00df2a9b3dbc + a=ice-ufrag:4ZcD + a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 - a=setup:passive + C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4: + 0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF + a=setup:actpass + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid + + m=video 0 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 0.0.0.0 + a=mid:v1 + a=sendrecv + a=rtpmap:100 VP8/90000 + a=rtpmap:101 rtx/90000 + a=fmtp:101 apt=100 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli + a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce + ac701365-eb06-42df-cc93-7f22bc308789 + a=bundle-only + |offer-C1-candidate-1| looks like: - m=video 52546 UDP/TLS/RTP/SAVPF 100 101 - c=IN IP4 11.22.33.44 - a=rtcp:52546 IN IP4 11.22.33.44 - a=mid:v2 - a=recvonly + ufrag 4ZcD + index 0 + mid a1 + attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay + raddr 0.0.0.0 rport 0 + + The SDP for |answer-C1| looks like: + + v=0 + o=- 6386516489780559513 1 IN IP4 0.0.0.0 + s=- + t=0 0 + a=ice-options:trickle + a=group:BUNDLE a1 v1 + a=group:LS a1 v1 + + m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 0.0.0.0 + a=mid:a1 + a=sendonly + a=rtpmap:96 opus/48000/2 + a=rtpmap:0 PCMU/8000 + a=rtpmap:8 PCMA/8000 + a=rtpmap:97 telephone-event/8000 + a=rtpmap:98 telephone-event/48000 + a=maxptime:120 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:751f239e-4ae0-c549-aa3d-890de772998b + 04b5a445-82cc-c9e8-9ffe-c24d0ef4b0ff + a=ice-ufrag:TpaA + a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/ + a=fingerprint:sha-256 + A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC: + 3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D + a=setup:active + a=dtls-id:1 + a=rtcp-mux + a=rtcp-mux-only + a=rtcp-rsize + + m=video 9 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 0.0.0.0 + a=mid:v1 + a=sendonly a=rtpmap:100 VP8/90000 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtcp-fb:100 ccm fir + a=rtcp-fb:100 nack + a=rtcp-fb:100 nack pli + a=msid:751f239e-4ae0-c549-aa3d-890de772998b + 39292672-c102-d075-f580-5826f31ca958 + + |answer-C1-candidate-1| looks like: + + ufrag TpaA + index 0 + mid a1 + attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay + raddr 0.0.0.0 rport 0 + + The SDP for |offer-C2| looks like: + + v=0 + o=- 6386516489780559513 2 IN IP4 0.0.0.0 + s=- + t=0 0 + a=ice-options:trickle + a=group:BUNDLE a1 v1 + a=group:LS a1 v1 + + m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 192.0.2.200 + a=mid:a1 + a=sendrecv + a=rtpmap:96 opus/48000/2 + a=rtpmap:0 PCMU/8000 + a=rtpmap:8 PCMA/8000 + a=rtpmap:97 telephone-event/8000 + a=rtpmap:98 telephone-event/48000 + a=maxptime:120 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:751f239e-4ae0-c549-aa3d-890de772998b + 04b5a445-82cc-c9e8-9ffe-c24d0ef4b0ff + a=ice-ufrag:TpaA + a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/ a=fingerprint:sha-256 - 19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04 - :BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2 + A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC: + 3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D + a=setup:actpass + a=dtls-id:1 + a=rtcp-mux + a=rtcp-mux-only + a=rtcp-rsize + a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay + raddr 0.0.0.0 rport 0 + a=end-of-candidates + m=video 12200 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 192.0.2.200 + a=mid:v1 + a=sendrecv + a=rtpmap:100 VP8/90000 + a=rtpmap:101 rtx/90000 + a=fmtp:101 apt=100 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=rtcp-fb:100 ccm fir + a=rtcp-fb:100 nack + a=rtcp-fb:100 nack pli + a=msid:751f239e-4ae0-c549-aa3d-890de772998b + 39292672-c102-d075-f580-5826f31ca958 + + The SDP for |answer-C2| looks like: + + v=0 + o=- 1070771854436052752 2 IN IP4 0.0.0.0 + s=- + t=0 0 + a=ice-options:trickle + a=group:BUNDLE a1 v1 + a=group:LS a1 v1 + m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98 + c=IN IP4 192.0.2.100 + a=mid:a1 + a=sendrecv + a=rtpmap:96 opus/48000/2 + a=rtpmap:0 PCMU/8000 + a=rtpmap:8 PCMA/8000 + a=rtpmap:97 telephone-event/8000 + a=rtpmap:98 telephone-event/48000 + a=maxptime:120 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid + a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level + a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce + e80098db-7159-3c06-229a-00df2a9b3dbc + a=ice-ufrag:4ZcD + a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD + a=fingerprint:sha-256 + C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4: + 0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF a=setup:passive + a=dtls-id:1 a=rtcp-mux + a=rtcp-mux-only a=rtcp-rsize - a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:mid + a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay + raddr 0.0.0.0 rport 0 + a=end-of-candidates + + m=video 12100 UDP/TLS/RTP/SAVPF 100 101 + c=IN IP4 192.0.2.100 + a=mid:v1 + a=sendrecv + a=rtpmap:100 VP8/90000 + a=rtpmap:101 rtx/90000 + a=fmtp:101 apt=100 + a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli + a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce + ac701365-eb06-42df-cc93-7f22bc308789 8. Security Considerations The IETF has published separate documents [I-D.ietf-rtcweb-security-arch] [I-D.ietf-rtcweb-security] describing the security architecture for WebRTC as a whole. The remainder of this section describes security considerations for this document. While formally the JSEP interface is an API, it is better to think of it is an Internet protocol, with the JS being untrustworthy from the - perspective of the browser. Thus, the threat model of [RFC3552] + perspective of the endpoint. Thus, the threat model of [RFC3552] applies. In particular, JS can call the API in any order and with any inputs, including malicious ones. This is particularly relevant when we consider the SDP which is passed to setLocalDescription(). While correct API usage requires that the application pass in SDP which was derived from createOffer() or createAnswer(), there is no - guarantee that applications do so. The browser MUST be prepared for - the JS to pass in bogus data instead. + guarantee that applications do so. The JSEP implementation MUST be + prepared for the JS to pass in bogus data instead. Conversely, the application programmer MUST recognize that the JS - does not have complete control of browser behavior. One case that + does not have complete control of endpoint behavior. One case that bears particular mention is that editing ICE candidates out of the SDP or suppressing trickled candidates does not have the expected behavior: implementations will still perform checks from those candidates even if they are not sent to the other side. Thus, for instance, it is not possible to prevent the remote peer from learning your public IP address by removing server reflexive candidates. + Applications which wish to conceal their public IP address should instead configure the ICE agent to use only relay candidates. 9. IANA Considerations This document requires no actions from IANA. 10. Acknowledgements - Significant text incorporated in the draft as well and review was - provided by Peter Thatcher, Taylor Brandstetter, Harald Alvestrand - and Suhas Nandakumar. Dan Burnett, Neil Stratford, Anant Narayanan, - Andrew Hutton, Richard Ejzak, Adam Bergkvist and Matthew Kaufman all + Harald Alvestrand, Taylor Brandstetter, Suhas Nandakumar, and Peter + Thatcher provided significant text for this draft. Bernard Aboba, + Adam Bergkvist, Dan Burnett, Ben Campbell, Alissa Cooper, Richard + Ejzak, Stefan Hakansson, Ted Hardie, Christer Holmberg Andrew Hutton, + Randell Jesup, Matthew Kaufman, Anant Narayanan, Adam Roach, Neil + Stratford, Martin Thomson, Sean Turner, and Magnus Westerlund all provided valuable feedback on this proposal. 11. References 11.1. Normative References [I-D.ietf-avtcore-rtp-multi-stream] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, "Sending Multiple RTP Streams in a Single RTP Session", draft-ietf-avtcore-rtp-multi-stream-11 (work in progress), @@ -3961,31 +4392,56 @@ [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, June 2010. [RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)", RFC 6236, May 2011. [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security Version 1.2", RFC 6347, January 2012. + [RFC6716] Valin, JM., Vos, K., and T. Terriberry, "Definition of the + Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716, + September 2012, . + [RFC6904] Lennox, J., "Encryption of Header Extensions in the Secure Real-time Transport Protocol (SRTP)", RFC 6904, April 2013. + [RFC7160] Petit-Huguenin, M. and G. Zorn, Ed., "Support for Multiple + Clock Rates in an RTP Session", RFC 7160, + DOI 10.17487/RFC7160, April 2014, + . + + [RFC7587] Spittka, J., Vos, K., and JM. Valin, "RTP Payload Format + for the Opus Speech and Audio Codec", RFC 7587, + DOI 10.17487/RFC7587, June 2015, + . + [RFC7850] Nandakumar, S., "Registering Values of the SDP 'proto' Field for Transporting RTP Media over TCP under Various RTP Profiles", RFC 7850, DOI 10.17487/RFC7850, April 2016, . + [RFC7941] Westerlund, M., Burman, B., Even, R., and M. Zanaty, "RTP + Header Extension for the RTP Control Protocol (RTCP) + Source Description Items", RFC 7941, DOI 10.17487/RFC7941, + August 2016, . + 11.2. Informative References + [I-D.ietf-avtext-lrr] + Lennox, J., Hong, D., Uberti, J., Homer, S., and M. + Flodman, "The Layer Refresh Request (LRR) RTCP Feedback + Message", draft-ietf-avtext-lrr-03 (work in progress), + July 2016. + [I-D.ietf-rtcweb-ip-handling] Uberti, J. and G. Shieh, "WebRTC IP Address Handling Recommendations", draft-ietf-rtcweb-ip-handling-01 (work in progress), March 2016. [I-D.nandakumar-rtcweb-sdp] Nandakumar, S. and C. Jennings, "SDP for the WebRTC", draft-nandakumar-rtcweb-sdp-02 (work in progress), July 2013. @@ -3994,37 +4450,47 @@ [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, July 2003, . [RFC3556] Casner, S., "Session Description Protocol (SDP) Bandwidth Modifiers for RTP Control Protocol (RTCP) Bandwidth", RFC 3556, July 2003. + [RFC3611] Friedman, T., Caceres, R., and A. Clark, "RTP Control + Protocol Extended Reports (RTCP XR)", RFC 3611, + DOI 10.17487/RFC3611, November 2003, + . + [RFC3960] Camarillo, G. and H. Schulzrinne, "Early Media and Ringing Tone Generation in the Session Initiation Protocol (SIP)", RFC 3960, December 2004. [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session Description Protocol (SDP) Security Descriptions for Media Streams", RFC 4568, July 2006. [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. Hakenberg, "RTP Retransmission Payload Format", RFC 4588, July 2006. [RFC4733] Schulzrinne, H. and T. Taylor, "RTP Payload for DTMF Digits, Telephony Tones, and Telephony Signals", RFC 4733, DOI 10.17487/RFC4733, December 2006, . + [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, + "Codec Control Messages in the RTP Audio-Visual Profile + with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104, + February 2008, . + [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size Real-Time Transport Control Protocol (RTCP): Opportunities and Consequences", RFC 5506, April 2009. [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific Media Attributes in the Session Description Protocol (SDP)", RFC 5576, June 2009. [RFC5763] Fischl, J., Tschofenig, H., and E. Rescorla, "Framework for Establishing a Secure Real-time Transport Protocol @@ -4034,26 +4500,38 @@ [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer Security (DTLS) Extension to Establish Keys for the Secure Real-time Transport Protocol (SRTP)", RFC 5764, May 2010. [RFC6464] Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time Transport Protocol (RTP) Header Extension for Client-to- Mixer Audio Level Indication", RFC 6464, DOI 10.17487/RFC6464, December 2011, . + [RFC6544] Rosenberg, J., Keranen, A., Lowekamp, B., and A. Roach, + "TCP Candidates with Interactive Connectivity + Establishment (ICE)", RFC 6544, DOI 10.17487/RFC6544, + March 2012, . + [RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms for Real-Time Transport Protocol (RTP) Sources", RFC 7656, DOI 10.17487/RFC7656, November 2015, . + [TS26.114] + 3GPP TS 26.114 V12.8.0, "3rd Generation Partnership + Project; Technical Specification Group Services and System + Aspects; IP Multimedia Subsystem (IMS); Multimedia + Telephony; Media handling and interaction (Release 12)", + December 2014, . + [W3C.WD-webrtc-20140617] Bergkvist, A., Burnett, D., Narayanan, A., and C. Jennings, "WebRTC 1.0: Real-time Communication Between Browsers", World Wide Web Consortium WD WD-webrtc- 20140617, June 2014, . Appendix A. Appendix A For the syntax validation performed in Section 5.7, the following @@ -4097,29 +4575,31 @@ +-----------------------+-------------------------------------------+ Table 1: SDP ABNF References Appendix B. Appendix B The following text is meant to completely replace section "Associating RTP/RTCP Streams With Correct SDP Media Description" of [I-D.ietf-mmusic-sdp-bundle-negotiation]. - As described in [RFC3550], RTP/RTCP packets are associated with RTP - streams as defined in [RFC7656]. Each RTP stream is identified by an - SSRC value, and each RTP/RTCP packet carries an SSRC value that is - used to associate the packet with the correct RTP stream. An RTCP - packet can carry multiple SSRC values, and might therefore be - associated with multiple RTP streams. + As described in [RFC3550], RTP packets are associated with RTP + streams [RFC7656]. Each RTP stream is identified by an SSRC value, + and each RTP packet includes an SSRC field that is used to associate + the packet with the correct RTP stream. RTCP packets also use SSRCs + to identify which RTP streams the packet relates to. However, a RTCP + packet can contain multiple SSRC fields, in the course of providing + feedback or reports on different RTP streams, and therefore can be + associated with multiple such streams. - In order to be able to process received RTP/RTCP packets correctly it - must be possible to associate an RTP stream with the correct "m=" + In order to be able to process received RTP/RTCP packets correctly, + it must be possible to associate an RTP stream with the correct "m=" line, as the "m=" line and SDP attributes associated with the "m=" line contain information needed to process the packets. As all RTP streams associated with a BUNDLE group use the same address:port combination for sending and receiving RTP/RTCP packets, the local address:port combination cannot be used to associate an RTP stream with the correct "m=" line. In addition, multiple RTP streams might be associated with the same "m=" line. An offerer and answerer can inform each other which SSRC values they @@ -4129,126 +4609,264 @@ that information. Due to this, before the offerer has received the answer, the offerer will not be able to associate an RTP stream with the correct "m=" line using the SSRC value associated with the RTP stream. In addition, the offerer and answerer may start using new SSRC values mid-session, without informing each other using the SDP 'ssrc' attribute. In order for an offerer and answerer to always be able to associate an RTP stream with the correct "m=" line, the offerer and answerer using the BUNDLE extension MUST support the mechanism defined in - [I-D.ietf-mmusic-sdp-bundle-negotiation] section 14. where the + [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 14, where the offerer and answerer insert the identification-tag associated with an "m=" line (provided by the remote peer) into RTP and RTCP packets associated with a BUNDLE group. - The mapping from an SSRC to an identification-tag is carried in RTCP - SDES packets or in RTP header extensions - ([I-D.ietf-mmusic-sdp-bundle-negotiation] section 14). Since a - compound RTCP packet can contain multiple RTCP SDES packets, and each - RTCP SDES packet can contain multiple chunks, an RTCP packet can - contain several SSRC to identification-tag mappings. The offerer and - answerer maintain tables used for routing that are updated each time - an RTP/RTCP packet contains new information that affects how packets - should be routed. + When using this mechanism, the mapping from an SSRC to an + identification-tag is carried in RTP header extensions or RTCP SDES + packets, as specified in [I-D.ietf-mmusic-sdp-bundle-negotiation], + Section 14). Since a compound RTCP packet can contain multiple RTCP + SDES packets, and each RTCP SDES packet can contain multiple chunks, + a single RTCP packet can contain several SSRC to identification-tag + mappings. The offerer and answerer maintain tables used for routing + that are updated each time an RTP/RTCP packet contains new + information that affects how packets should be routed. - To prepare for demultiplexing RTP packets to the correct "m=" line, - the following steps MUST be followed for each BUNDLE group. + However, some implementations of + [I-D.ietf-mmusic-sdp-bundle-negotiation] may not include this + identification-tag in their RTP and RTCP traffic when using BUNDLE, + and instead use a payload type based mechanism for demuxing. In this + situation, each "m=" line MUST use unique payload type values, in + order for the payload type to be a reliable indicator of the relevant + "m=" line for the RTP stream. + + Applications can implement RTP stacks in many different ways. The + algorithm below details one way that demultiplexing can be + accomplished, but is not meant to be prescriptive about exactly how + an RTP stack needs to be implemented. Applications MAY use any + algorithm that achieves equivalent results to those described in the + algorithm below. + + To prepare for demultiplexing RTP/RTCP packets to the correct "m=" + line, the following steps MUST be followed for each BUNDLE group. Construct a table mapping MID to "m=" line for each "m=" line in this BUNDLE group. Note that an "m=" line may only have one MID. Construct a table mapping incoming SSRC to "m=" line for each "m=" line in this BUNDLE group and for each SSRC configured for receiving in that "m=" line. Construct a table mapping outgoing SSRC to "m=line" for each "m=" line in this BUNDLE group and for each SSRC configured for sending in that "m=" line. Construct a table mapping payload type to "m=" line for each "m=" line in the BUNDLE group and for each payload type configured for receiving in that "m=" line. If any payload type is configured for receiving in more than one "m=" line in the BUNDLE group, do - not it include it in the table. + not it include it in the table, as it cannot be used to uniquely + identify a "m=" line. Note that for each of these tables, there can only be one mapping for any given key (MID, SSRC, or PT). In other words, the tables are not multimaps. As "m=" lines are added or removed from the BUNDLE groups, or their configurations are changed, the tables above MUST also be updated. For each RTP packet received, the following steps MUST be followed to route the packet to the correct "m=" section within a BUNDLE group. Note that the phrase 'deliver a packet to the "m=" line' means to further process the packet as would normally happen with RTP/RTCP, if it were received on a transport associated with that "m=" line outside of a BUNDLE group (i.e., if the "m=" line were not BUNDLEd), including dropping an RTP packet if the packet's PT does not match any PT in the "m=" line. - If the packet has a MID and that MID is not in the table mapping + If the packet has a MID, and that MID is not in the table mapping MID to "m=" line, drop the packet and stop. - If the packet has a MID and that MID is in the table mapping MID - to "m=" line, update the incoming SSRC mapping table to include an - entry that maps the packet's SSRC to the "m=" line for that MID. + If the packet has a MID, and the packet's extended sequence number + is greater than that of the last MID update, as discussed in + [RFC7941], Section 4.2.6, update the incoming SSRC mapping table + to include an entry that maps the packet's SSRC to the "m=" line + for that MID. - If the packet's SSRC is in the incoming SSRC mapping table, route - the packet to the associated "m=" line and stop. + If the packet's SSRC is in the incoming SSRC mapping table, check + that the packet's PT matches a PT included on the associated "m=" + line. If so, route the packet to that associated "m=" line and + stop; otherwise drop the packet and stop. If the packet's payload type is in the payload type table, update the the incoming SSRC mapping table to include an entry that maps the packet's SSRC to the "m=" line for that payload type. In addition, route the packet to the associated "m=" line and stop. Otherwise, drop the packet. For each RTCP packet received (including each RTCP packet that is part of a compound RTCP packet), the packet MUST be routed to the - appropriate handler for the SSRCs it contains information about. - Some examples of such handling are given below. + "m=" line for the RTP streams it contains information about. This + routing is type-dependent, as each kind of RTCP packet has its own + mechanism for associating it with the relevant RTP streams. - If the packet is of type SR, and the sender SSRC for the packet is + Packets for which no appropriate "m=" line can be identified (i.e., + for unknown RTP streams) are not relevant in the context of this + algorithm and MAY be dropped. This situation may occur with certain + multiparty RTP topologies. + + Rules for handling the various types of RTCP packets are explained + below. + + If the packet is of type SDES, for each chunk in the packet whose + SSRC is found in the incoming SSRC table, deliver a copy of the + packet to the "m=" line associated with that SSRC. In addition, + for any SDES MID items contained in these chunks, if the MID is + found in the table mapping MID to "m=" line, update the incoming + SSRC table to include an entry that maps the chunk's SSRC to the + "m=" line associated with that MID, unless the packet is older + than the packet that most recently updated the mapping for this + SSRC, as discussed in [RFC7941], Section 4.2.6. + + Note that if an SDES packet is received as part of a compound RTCP + packet, the SSRC to "m=" line mapping may not exist until the SDES + packet is handled (e.g., in the case where RTCP for a source is + received before any RTP packets). Therefore, when processing a + compound packet, any contained SDES packet MUST be handled first. + + If the packet is of type BYE, it indicates that the RTP streams + referenced in the packet are ending. Therefore, for each SSRC + indicated in the packet that is found in the incoming SSRC table, + first deliver a copy of the packet to the "m=" line associated + with that SSRC, but then remove the entry for that SSRC from the + incoming SSRC table. + + If the packet is of type SR or RR, for each report block in the + report whose "SSRC of source" is found in the outgoing SSRC table, + deliver a copy of the RTCP packet to the "m=" line associated with + that SSRC. In addition, if the packet is of type SR, and the + sender SSRC for the packet is found in the incoming SSRC table, + deliver a copy of the packet to the "m=" line associated with that + SSRC. + + If the implementation supports RTCP XR and the packet is of type + XR, as defined in [RFC3611], for each report block in the report + whose "SSRC of source" is is found in the outgoing SSRC table, + deliver a copy of the RTCP packet to the "m=" line associated with + that SSRC. In addition, if the sender SSRC for the packet is found in the incoming SSRC table, deliver a copy of the packet to - the "m=" line associated with that SSRC. In addition, for each - report block in the report whose SSRC is found in the outgoing - SSRC table, deliver a copy of the RTCP packet to the "m=" line - associated with that SSRC. + the "m=" line associated with that SSRC. - If the packet is of type RR, for each report block in the packet - whose SSRC is found in the outgoing SSRC table, deliver a copy of - the RTCP packet to the "m=" line associated with that SSRC. + If the packet is a feedback message of type RTPFB or PSFB, as + defined in [RFC4585], it will contain a media source SSRC, and + this SSRC is used for routing certain subtypes of feedback + messages. However, several subtypes of PSFB messages include + target SSRC(s) in a section called Feedback Control Information + (FCI). For these messages, the target SSRC(s) are used for + routing. - If the packet is of type SDES, and the sender SSRC for the packet - is found in the incoming SSRC table, deliver the packet to the - "m=" line associated with that SSRC. In addition, for each chunk - in the packet that contains a MID that is in the table mapping MID - to "m=" line, update the incoming SSRC mapping table to include an - entry that maps the SSRC for that chunk to the "m=" line - associated with that MID. (This case can occur when RTCP for a - source is received before any RTP packets.) + If the packet is a feedback message that does not include target + SSRCs in its FCI section, and the media source SSRC is found in + the outgoing SSRC table, deliver the packet to the "m=" line + associated with that SSRC. RTPFB and PSFB types that are handled + in this way include: - If the packet is of type BYE, for each SSRC indicated in the - packet that is found in the incoming SSRC table, deliver a copy of - the packet to the "m=" line associated with that SSRC. + Generic NACK: [RFC4585] (PT=RTPFB, FMT=1). - If the packet is of type RTPFB or PSFB, as defined in [RFC4585], - and the media source SSRC for the packet is found in the outgoing - SSRC table, deliver the packet to the "m=" line associated with - that SSRC. + Picture Loss Indication (PLI): [RFC4585] (PT=PSFB, FMT=1). + + Slice Loss Indication (SLI): [RFC4585] (PT=PSFB, FMT=2). + + Reference Picture Selection Indication (RPSI): [RFC4585] + (PT=PSFB, FMT=3). + + If the packet is a feedback message that does include target + SSRC(s) in its FCI section, it can either be a request or a + notification. Requests reference a RTP stream that is being sent + by the message recipient, whereas notifications are responses to + an earlier request, and therefore reference a RTP stream that is + being received by the message recipient. + + If the packet is a feedback request that includes target SSRC(s), + for each target SSRC that is found in the outgoing SSRC table, + deliver a copy of the RTCP packet to the "m=" line associated with + that SSRC. PSFB types that are handled in this way include: + + Full Intra Request (FIR): [RFC5104] (PT=PSFB, FMT=4). + + Temporal-Spatial Trade-off Request (TSTR): [RFC5104] (PT=PSFB, + FMT=5). + + H.271 Video Back Channel Message (VBCM): [RFC5104] + (PT=PSFB, FMT=7). + + Layer Refresh Request (LRR): [I-D.ietf-avtext-lrr] (PT=PSFB, + FMT=TBD). + + If the packet is a feedback notification that include target + SSRC(s), for each target SSRC that is found in the incoming SSRC + table, deliver a copy of the RTCP packet to the "m=" line + associated with that SSRC. PSFB types that are handled in this + way include: + + Temporal-Spatial Trade-off Notification (TSTN): [RF + C5104] (PT=PSFB, FMT=6). This message is a notification in + response to a prior TSTR. + + If the packet is of type APP, the only routing information + included is the source of the packet, and therefore the packet + could be related to any existing "m=" line. Accordingly, deliver + a copy of the packet to each "m=" line. Appendix C. Change log Note: This section will be removed by RFC Editor before publication. + Changes in draft-19: + + o Examples are now machine-generated for correctness, and use IETF- + approved example IP addresses. + + o Add early transport warmup example, and add missing attributes to + existing examples. + + o Only send "a=rtcp-mux-only" and "a=bundle-only" on new m= + sections. + + o Update references. + + o Add coverage of a=identity. + + o Explain the lipsync group algorithm more thoroughly. + + o Remove unnecessary list of MTI specs. + + o Allow codecs which weren't offered to appear in answers and which + weren't selected to appear in subsequent offers. + + o Codec preferences now are applied on both initial and subsequent + offers and answers. + + o Clarify a=msid handling for recvonly m= sections. + + o Clarify behavior of attributes for bundle-only data channels. + + o Allow media attributes to appear in data m= sections when all the + media m= sections are bundle-only. + + o Use consistent terminology for JSEP implementations. + + o Describe how to handle failed API calls. + + o Some cleanup on routing rules. + Changes in draft-18: o Update demux algorithm and move it to an appendix in preparation for merging it into BUNDLE. o Clarify why we can't handle an incoming offer to send simulcast. o Expand IceCandidate object text. o Further document use of ICE candidate pool.