--- 1/draft-ietf-rtcweb-jsep-17.txt 2017-01-16 16:13:10.211390883 -0800 +++ 2/draft-ietf-rtcweb-jsep-18.txt 2017-01-16 16:13:10.407395401 -0800 @@ -1,21 +1,21 @@ Network Working Group J. Uberti Internet-Draft Google Intended status: Standards Track C. Jennings -Expires: April 24, 2017 Cisco +Expires: July 20, 2017 Cisco E. Rescorla, Ed. Mozilla - October 21, 2016 + January 16, 2017 Javascript Session Establishment Protocol - draft-ietf-rtcweb-jsep-17 + draft-ietf-rtcweb-jsep-18 Abstract This document describes the mechanisms for allowing a Javascript application to control the signaling plane of a multimedia session via the interface specified in the W3C RTCPeerConnection API, and discusses how this relates to existing signaling protocols. Status of This Memo @@ -25,25 +25,25 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on April 24, 2017. + This Internet-Draft will expire on July 20, 2017. Copyright Notice - Copyright (c) 2016 IETF Trust and the persons identified as the + Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as @@ -62,91 +62,95 @@ 3.4. Session Description Control . . . . . . . . . . . . . . . 10 3.4.1. RtpTransceivers . . . . . . . . . . . . . . . . . . . 10 3.4.2. RtpSenders . . . . . . . . . . . . . . . . . . . . . 11 3.4.3. RtpReceivers . . . . . . . . . . . . . . . . . . . . 11 3.5. ICE . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.5.1. ICE Gathering Overview . . . . . . . . . . . . . . . 11 3.5.2. ICE Candidate Trickling . . . . . . . . . . . . . . . 12 3.5.2.1. ICE Candidate Format . . . . . . . . . . . . . . 12 3.5.3. ICE Candidate Policy . . . . . . . . . . . . . . . . 13 3.5.4. ICE Candidate Pool . . . . . . . . . . . . . . . . . 14 - 3.6. Video Size Negotiation . . . . . . . . . . . . . . . . . 14 + 3.6. Video Size Negotiation . . . . . . . . . . . . . . . . . 15 3.6.1. Creating an imageattr Attribute . . . . . . . . . . . 15 3.6.2. Interpreting an imageattr Attribute . . . . . . . . . 16 3.7. Simulcast . . . . . . . . . . . . . . . . . . . . . . . . 17 3.8. Interactions With Forking . . . . . . . . . . . . . . . . 18 - 3.8.1. Sequential Forking . . . . . . . . . . . . . . . . . 18 + 3.8.1. Sequential Forking . . . . . . . . . . . . . . . . . 19 3.8.2. Parallel Forking . . . . . . . . . . . . . . . . . . 19 4. Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.1. PeerConnection . . . . . . . . . . . . . . . . . . . . . 20 4.1.1. Constructor . . . . . . . . . . . . . . . . . . . . . 20 4.1.2. addTrack . . . . . . . . . . . . . . . . . . . . . . 22 - 4.1.3. addTransceiver . . . . . . . . . . . . . . . . . . . 22 - 4.1.4. createDataChannel . . . . . . . . . . . . . . . . . . 23 - 4.1.5. createOffer . . . . . . . . . . . . . . . . . . . . . 23 - 4.1.6. createAnswer . . . . . . . . . . . . . . . . . . . . 24 - 4.1.7. SessionDescriptionType . . . . . . . . . . . . . . . 25 - 4.1.7.1. Use of Provisional Answers . . . . . . . . . . . 25 - 4.1.7.2. Rollback . . . . . . . . . . . . . . . . . . . . 26 - 4.1.8. setLocalDescription . . . . . . . . . . . . . . . . . 27 - 4.1.9. setRemoteDescription . . . . . . . . . . . . . . . . 28 - 4.1.10. currentLocalDescription . . . . . . . . . . . . . . . 28 - 4.1.11. pendingLocalDescription . . . . . . . . . . . . . . . 28 - 4.1.12. currentRemoteDescription . . . . . . . . . . . . . . 28 - 4.1.13. pendingRemoteDescription . . . . . . . . . . . . . . 29 - 4.1.14. canTrickleIceCandidates . . . . . . . . . . . . . . . 29 - 4.1.15. setConfiguration . . . . . . . . . . . . . . . . . . 30 - 4.1.16. addIceCandidate . . . . . . . . . . . . . . . . . . . 30 - 4.2. RtpTransceiver . . . . . . . . . . . . . . . . . . . . . 31 - 4.2.1. stop . . . . . . . . . . . . . . . . . . . . . . . . 31 - 4.2.2. stopped . . . . . . . . . . . . . . . . . . . . . . . 31 - 4.2.3. setDirection . . . . . . . . . . . . . . . . . . . . 31 - 4.2.4. setCodecPreferences . . . . . . . . . . . . . . . . . 32 - 5. SDP Interaction Procedures . . . . . . . . . . . . . . . . . 32 - 5.1. Requirements Overview . . . . . . . . . . . . . . . . . . 32 - 5.1.1. Implementation Requirements . . . . . . . . . . . . . 33 - 5.1.2. Usage Requirements . . . . . . . . . . . . . . . . . 34 - 5.1.3. Profile Names and Interoperability . . . . . . . . . 34 - 5.2. Constructing an Offer . . . . . . . . . . . . . . . . . . 35 - 5.2.1. Initial Offers . . . . . . . . . . . . . . . . . . . 35 - 5.2.2. Subsequent Offers . . . . . . . . . . . . . . . . . . 41 - 5.2.3. Options Handling . . . . . . . . . . . . . . . . . . 44 - 5.2.3.1. IceRestart . . . . . . . . . . . . . . . . . . . 44 - 5.2.3.2. VoiceActivityDetection . . . . . . . . . . . . . 45 - 5.3. Generating an Answer . . . . . . . . . . . . . . . . . . 45 - 5.3.1. Initial Answers . . . . . . . . . . . . . . . . . . . 45 - 5.3.2. Subsequent Answers . . . . . . . . . . . . . . . . . 50 - 5.3.3. Options Handling . . . . . . . . . . . . . . . . . . 51 - 5.3.3.1. VoiceActivityDetection . . . . . . . . . . . . . 51 - 5.4. Modifying an Offer or Answer . . . . . . . . . . . . . . 51 - 5.5. Processing a Local Description . . . . . . . . . . . . . 52 - 5.6. Processing a Remote Description . . . . . . . . . . . . . 53 - 5.7. Parsing a Session Description . . . . . . . . . . . . . . 53 - 5.7.1. Session-Level Parsing . . . . . . . . . . . . . . . . 54 - 5.7.2. Media Section Parsing . . . . . . . . . . . . . . . . 55 - 5.7.3. Semantics Verification . . . . . . . . . . . . . . . 58 - 5.8. Applying a Local Description . . . . . . . . . . . . . . 59 - 5.9. Applying a Remote Description . . . . . . . . . . . . . . 60 - 5.10. Applying an Answer . . . . . . . . . . . . . . . . . . . 64 - 6. Processing RTP/RTCP packets . . . . . . . . . . . . . . . . . 66 + 4.1.3. removeTrack . . . . . . . . . . . . . . . . . . . . . 23 + 4.1.4. addTransceiver . . . . . . . . . . . . . . . . . . . 23 + 4.1.5. createDataChannel . . . . . . . . . . . . . . . . . . 23 + 4.1.6. createOffer . . . . . . . . . . . . . . . . . . . . . 24 + 4.1.7. createAnswer . . . . . . . . . . . . . . . . . . . . 25 + 4.1.8. SessionDescriptionType . . . . . . . . . . . . . . . 25 + 4.1.8.1. Use of Provisional Answers . . . . . . . . . . . 26 + 4.1.8.2. Rollback . . . . . . . . . . . . . . . . . . . . 27 + 4.1.9. setLocalDescription . . . . . . . . . . . . . . . . . 28 + 4.1.10. setRemoteDescription . . . . . . . . . . . . . . . . 28 + 4.1.11. currentLocalDescription . . . . . . . . . . . . . . . 29 + 4.1.12. pendingLocalDescription . . . . . . . . . . . . . . . 29 + 4.1.13. currentRemoteDescription . . . . . . . . . . . . . . 29 + 4.1.14. pendingRemoteDescription . . . . . . . . . . . . . . 29 + 4.1.15. canTrickleIceCandidates . . . . . . . . . . . . . . . 30 + 4.1.16. setConfiguration . . . . . . . . . . . . . . . . . . 30 + 4.1.17. addIceCandidate . . . . . . . . . . . . . . . . . . . 31 + 4.2. RtpTransceiver . . . . . . . . . . . . . . . . . . . . . 32 + 4.2.1. stop . . . . . . . . . . . . . . . . . . . . . . . . 32 + 4.2.2. stopped . . . . . . . . . . . . . . . . . . . . . . . 32 + 4.2.3. setDirection . . . . . . . . . . . . . . . . . . . . 32 + 4.2.4. direction . . . . . . . . . . . . . . . . . . . . . . 32 + 4.2.5. currentDirection . . . . . . . . . . . . . . . . . . 33 + 4.2.6. setCodecPreferences . . . . . . . . . . . . . . . . . 33 + 5. SDP Interaction Procedures . . . . . . . . . . . . . . . . . 33 + 5.1. Requirements Overview . . . . . . . . . . . . . . . . . . 34 + 5.1.1. Implementation Requirements . . . . . . . . . . . . . 34 + 5.1.2. Usage Requirements . . . . . . . . . . . . . . . . . 35 + 5.1.3. Profile Names and Interoperability . . . . . . . . . 36 + 5.2. Constructing an Offer . . . . . . . . . . . . . . . . . . 37 + 5.2.1. Initial Offers . . . . . . . . . . . . . . . . . . . 37 + 5.2.2. Subsequent Offers . . . . . . . . . . . . . . . . . . 42 + 5.2.3. Options Handling . . . . . . . . . . . . . . . . . . 46 + 5.2.3.1. IceRestart . . . . . . . . . . . . . . . . . . . 46 + 5.2.3.2. VoiceActivityDetection . . . . . . . . . . . . . 46 + 5.3. Generating an Answer . . . . . . . . . . . . . . . . . . 47 + 5.3.1. Initial Answers . . . . . . . . . . . . . . . . . . . 47 + 5.3.2. Subsequent Answers . . . . . . . . . . . . . . . . . 51 + 5.3.3. Options Handling . . . . . . . . . . . . . . . . . . 53 + 5.3.3.1. VoiceActivityDetection . . . . . . . . . . . . . 53 + 5.4. Modifying an Offer or Answer . . . . . . . . . . . . . . 53 + 5.5. Processing a Local Description . . . . . . . . . . . . . 54 + 5.6. Processing a Remote Description . . . . . . . . . . . . . 54 + 5.7. Parsing a Session Description . . . . . . . . . . . . . . 55 + 5.7.1. Session-Level Parsing . . . . . . . . . . . . . . . . 55 + 5.7.2. Media Section Parsing . . . . . . . . . . . . . . . . 57 + 5.7.3. Semantics Verification . . . . . . . . . . . . . . . 59 + 5.8. Applying a Local Description . . . . . . . . . . . . . . 60 + 5.9. Applying a Remote Description . . . . . . . . . . . . . . 62 + 5.10. Applying an Answer . . . . . . . . . . . . . . . . . . . 65 + 6. Processing RTP/RTCP . . . . . . . . . . . . . . . . . . . . . 68 7. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7.1. Simple Example . . . . . . . . . . . . . . . . . . . . . 68 7.2. Normal Examples . . . . . . . . . . . . . . . . . . . . . 72 8. Security Considerations . . . . . . . . . . . . . . . . . . . 81 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 81 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 81 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 82 11.1. Normative References . . . . . . . . . . . . . . . . . . 82 11.2. Informative References . . . . . . . . . . . . . . . . . 85 - Appendix A. Appendix A . . . . . . . . . . . . . . . . . . . . . 86 - Appendix B. Change log . . . . . . . . . . . . . . . . . . . . . 87 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 94 + Appendix A. Appendix A . . . . . . . . . . . . . . . . . . . . . 87 + Appendix B. Appendix B . . . . . . . . . . . . . . . . . . . . . 88 + Appendix C. Change log . . . . . . . . . . . . . . . . . . . . . 91 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 99 1. Introduction This document describes how the W3C WEBRTC RTCPeerConnection interface [W3C.WD-webrtc-20140617] is used to control the setup, management and teardown of a multimedia session. 1.1. General Design of JSEP The thinking behind WebRTC call setup has been to fully specify and @@ -170,25 +174,25 @@ In this document, the use of JSEP is described as if it always occurs between two browsers. Note though in many cases it will actually be between a browser and some kind of server, such as a gateway or MCU. This distinction is invisible to the browser; it just follows the instructions it is given via the API. JSEP's handling of session descriptions is simple and straightforward. Whenever an offer/answer exchange is needed, the initiating side creates an offer by calling a createOffer() API. The - application optionally modifies that offer, and then uses it to set - up its local config via the setLocalDescription() API. The offer is - then sent off to the remote side over its preferred signaling - mechanism (e.g., WebSockets); upon receipt of that offer, the remote - party installs it using the setRemoteDescription() API. + application then uses that offer to set up its local config via the + setLocalDescription() API. The offer is finally sent off to the + remote side over its preferred signaling mechanism (e.g., + WebSockets); upon receipt of that offer, the remote party installs it + using the setRemoteDescription() API. To complete the offer/answer exchange, the remote party uses the createAnswer() API to generate an appropriate answer, applies it using the setLocalDescription() API, and sends the answer back to the initiator over the signaling channel. When the initiator gets that answer, it installs it using the setRemoteDescription() API, and initial setup is complete. This process can be repeated for additional offer/answer exchanges. Regarding ICE [RFC5245], JSEP decouples the ICE state machine from @@ -353,23 +357,25 @@ JSEP also allows for an answer to be treated as provisional by the application. Provisional answers provide a way for an answerer to communicate initial session parameters back to the offerer, in order to allow the session to begin, while allowing a final answer to be specified later. This concept of a final answer is important to the offer/answer model; when such an answer is received, any extra resources allocated by the caller can be released, now that the exact session configuration is known. These "resources" can include things like extra ICE components, TURN candidates, or video decoders. - Provisional answers, on the other hand, do no such deallocation - results; as a result, multiple dissimilar provisional answers can be - received and applied during call setup. + Provisional answers, on the other hand, do no such deallocation; as a + result, multiple dissimilar provisional answers, with their own codec + choices, transport parameters, etc., can be received and applied + during call setup. Note that the final answer itself may be + different than any received provisional answers. In [RFC3264], the constraint at the signaling level is that only one offer can be outstanding for a given session, but at the media stack level, a new offer can be generated at any point. For example, when using SIP for signaling, if one offer is sent, then cancelled using a SIP CANCEL, another offer can be generated even though no answer was received for the first offer. To support this, the JSEP media layer can provide an offer via the createOffer() method whenever the Javascript application needs one for the signaling. The answerer can send back zero or more provisional answers, and finally end the @@ -415,28 +421,26 @@ setLocal(OFFER) setRemote(PRANSWER) Figure 2: JSEP State Machine Aside from these state transitions there is no other difference between the handling of provisional ("pranswer") and final ("answer") answers. 3.3. Session Description Format - In the WebRTC specification, session descriptions are formatted as - SDP messages. While this format is not optimal for manipulation from - Javascript, it is widely accepted, and frequently updated with new - features. Any alternate encoding of session descriptions would have - to keep pace with the changes to SDP, at least until the time that - this new encoding eclipsed SDP in popularity. As a result, JSEP - currently uses SDP as the internal representation for its session - descriptions. + JSEP's session descriptions use SDP syntax for their internal + representation. While this format is not optimal for manipulation + from Javascript, it is widely accepted, and frequently updated with + new features; any alternate encoding of session descriptions would + have to keep pace with the changes to SDP, at least until the time + that this new encoding eclipsed SDP in popularity. However, to simplify Javascript processing, and provide for future flexibility, the SDP syntax is encapsulated within a SessionDescription object, which can be constructed from SDP, and be serialized out to SDP. If future specifications agree on a JSON format for session descriptions, we could easily enable this object to generate and consume that JSON. Other methods may be added to SessionDescription in the future to simplify handling of SessionDescriptions from Javascript. In the @@ -463,70 +467,83 @@ RtpTransceivers allow the application to control the RTP media associated with one m= section. Each RtpTransceiver has an RtpSender and an RtpReceiver, which an application can use to control the sending and receiving of RTP media. The application may also modify the RtpTransceiver directly, for instance, by stopping it. RtpTransceivers generally have a 1:1 mapping with m= sections, although there may be more RtpTransceivers than m= sections when RtpTransceivers are created but not yet associated with a m= section, or if RtpTransceivers have been stopped and disassociated from m= - sections. An RtpTransceiver is never associated with more than one - m= section, and once a session description is applied, a m= section - is always associated with exactly one RtpTransceiver. + sections. An RtpTransceiver is said to be associated with an m= + section if its mid property is non-null; otherwise it is said to be + disassociated. The associated m= section is determined using a + mapping between transceivers and m= section indices, formed when + creating an offer or applying a remote offer. An RtpTransceiver is + never associated with more than one m= section, and once a session + description is applied, a m= section is always associated with + exactly one RtpTransceiver. RtpTransceivers can be created explicitly by the application or implicitly by calling setRemoteDescription with an offer that adds new m= sections. 3.4.2. RtpSenders RtpSenders allow the application to control how RTP media is sent. + An RtpSender is conceptually responsible for the outgoing RTP + stream(s) described by an m= section. This includes encoding the + attached MediaStreamTrack, sending RTP media packets, and generating/ + processing RTCP for the outgoing RTP streams(s). 3.4.3. RtpReceivers - RtpReceivers allows the application to control how RTP media is - received. + RtpReceivers allow the application to inspect how RTP media is + received. An RtpReceiver is conceptually responsible for the + incoming RTP stream(s) described by an m= section. This includes + processing received RTP media packets, decoding the incoming + stream(s) to produce a remote MediaStreamTrack, and generating/ + processing RTCP for the incoming RTP stream(s). 3.5. ICE 3.5.1. ICE Gathering Overview JSEP gathers ICE candidates as needed by the application. Collection of ICE candidates is referred to as a gathering phase, and this is - triggered either by the addition of a new or recycled m= line to the - local session description, or new ICE credentials in the description, - indicating an ICE restart. Use of new ICE credentials can be - triggered explicitly by the application, or implicitly by the browser - in response to changes in the ICE configuration. + triggered either by the addition of a new or recycled m= section to + the local session description, or new ICE credentials in the + description, indicating an ICE restart. Use of new ICE credentials + can be triggered explicitly by the application, or implicitly by the + browser in response to changes in the ICE configuration. When the ICE configuration changes in a way that requires a new gathering phase, a 'needs-ice-restart' bit is set. When this bit is set, calls to the createOffer API will generate new ICE credentials. This bit is cleared by a call to the setLocalDescription API with new ICE credentials from either an offer or an answer, i.e., from either a local- or remote-initiated ICE restart. When a new gathering phase starts, the ICE Agent will notify the application that gathering is occurring through an event. Then, when each new ICE candidate becomes available, the ICE Agent will supply it to the application via an additional event; these candidates will also automatically be added to the current and/or pending local session description. Finally, when all candidates have been gathered, an event will be dispatched to signal that the gathering process is complete. Note that gathering phases only gather the candidates needed by - new/recycled/restarting m= lines; other m= lines continue to use - their existing candidates. Also, when bundling is active, candidates - are only gathered (and exchanged) for the m= lines referenced in - BUNDLE-tags, as described in + new/recycled/restarting m= sections; other m= sections continue to + use their existing candidates. Also, when bundling is active, + candidates are only gathered (and exchanged) for the m= sections + referenced in BUNDLE-tags, as described in [I-D.ietf-mmusic-sdp-bundle-negotiation]. 3.5.2. ICE Candidate Trickling Candidate trickling is a technique through which a caller may incrementally provide candidates to the callee after the initial offer has been dispatched; the semantics of "Trickle ICE" are defined in [I-D.ietf-ice-trickle]. This process allows the callee to begin acting upon the call and setting up the ICE (and perhaps DTLS) connections immediately, without having to wait for the caller to @@ -542,62 +559,69 @@ applications that do not support this feature can simply wait for the indication that gathering is complete, and then create and send their offer, with all the candidates, at this time. Upon receipt of trickled candidates, the receiving application will supply them to its ICE Agent. This triggers the ICE Agent to start using the new remote candidates for connectivity checks. 3.5.2.1. ICE Candidate Format - As with session descriptions, the syntax of the IceCandidate object - provides some abstraction, but can be easily converted to and from - the SDP candidate lines. + In JSEP, ICE candidates are abstracted by an IceCandidate object, and + as with session descriptions, SDP syntax is used for the internal + representation. - The candidate lines are the only SDP information that is contained - within IceCandidate, as they represent the only information needed - that is not present in the initial offer (i.e., for trickle - candidates). This information is carried with the same syntax as the - "candidate-attribute" field defined for ICE. For example: + The candidate details are specified in an IceCandidate field, using + the same SDP syntax as the "candidate-attribute" field defined in + [RFC5245], Section 15.1. For example: candidate:1 1 UDP 1694498815 192.0.2.33 10000 typ host + + The IceCandidate object contains a field to indicate which ICE ufrag + it is associated with, as defined in [RFC5245], Section 15.4. This + value is used to determine which session description (and thereby + which gathering phase) this IceCandidate belongs to, which helps + resolve ambiguities during ICE restarts. If this field is absent in + a received IceCandidate (perhaps when communicating with a non-JSEP + endpoint), the most recently received session description is assumed. + The IceCandidate object also contains fields to indicate which m= - line it should be associated with. The m= line can be identified in - one of two ways; either by a m= line index, or a MID. The m= line - index is a zero-based index, with index N referring to the N+1th m= - line in the SDP sent by the entity which sent the IceCandidate. The - MID uses the "media stream identification" attribute, as defined in - [RFC5888], Section 4, to identify the m= line. JSEP implementations - creating an ICE Candidate object MUST populate both of these fields, - using the MID of the associated RtpTransceiver object (which may be - locally generated by the answerer when interacting with a non-JSEP - remote endpoint that does not support the MID attribute, as discussed - in Section 5.9 below). Implementations receiving an ICE Candidate - object MUST use the MID if present, or the m= line index, if not (the - non-JSEP remote endpoint case). + section it is associated with, which can be identified in one of two + ways, either by a m= section index, or a MID. The m= section index + is a zero-based index, with index N referring to the N+1th m= section + in the session description referenced by this IceCandidate. The MID + is a "media stream identification" value, as defined in [RFC5888], + Section 4, which provides a more robust way to identify the m= + section in the session description, using the MID of the associated + RtpTransceiver object (which may have been locally generated by the + answerer when interacting with a non-JSEP endpoint that does not + support the MID attribute, as discussed in Section 5.9 below). If + the MID field is present in a received IceCandidate, it MUST be used + for identification; otherwise, the m= section index is used instead. + + When creating an IceCandidate object, JSEP implementations MUST + populate all of these fields. 3.5.3. ICE Candidate Policy Typically, when gathering ICE candidates, the browser will gather all possible forms of initial candidates - host, server reflexive, and relay. However, in certain cases, applications may want to have more specific control over the gathering process, due to privacy or - related concerns. For example, one may want to suppress the use of - host candidates, to avoid exposing information about the local - network, or go as far as only using relay candidates, to leak as - little location information as possible (note that these choices come - with corresponding operational costs). To accomplish this, the - browser MUST allow the application to restrict which ICE candidates - are used in a session. Note that this filtering is applied on top of - any restrictions the browser chooses to enforce regarding which IP - addresses are permitted for the application, as discussed in - [I-D.ietf-rtcweb-ip-handling]. + related concerns. For example, one may want to only use relay + candidates, to leak as little location information as possible + (keeping in mind that this choice comes with corresponding + operational costs). To accomplish this, JSEP allows the application + to restrict which ICE candidates are used in a session. Note that + this filtering is applied on top of any restrictions the browser + chooses to enforce regarding which IP addresses are permitted for the + application, as discussed in [I-D.ietf-rtcweb-ip-handling]. There may also be cases where the application wants to change which types of candidates are used while the session is active. A prime example is where a callee may initially want to use only relay candidates, to avoid leaking location information to an arbitrary caller, but then change to use all candidates (for lower operational cost) once the user has indicated they want to take the call. For this scenario, the browser MUST allow the candidate policy to be changed in mid-session, subject to the aforementioned interactions with local policy. @@ -623,21 +646,23 @@ ahead of time, it may ask the browser to gather a pool of potential ICE candidates to help ensure rapid media setup. When setLocalDescription is eventually called, and the browser goes to gather the needed ICE candidates, it SHOULD start by checking if any candidates are available in the pool. If there are candidates in the pool, they SHOULD be handed to the application immediately via the ICE candidate event. If the pool becomes depleted, either because a larger-than-expected number of ICE components is used, or because the pool has not had enough time to gather candidates, the - remaining candidates are gathered as usual. + remaining candidates are gathered as usual. This only occurs for the + first offer/answer exchange, after which the candidate pool is + emptied and no longer used. One example of where this concept is useful is an application that expects an incoming call at some point in the future, and wants to minimize the time it takes to establish connectivity, to avoid clipping of initial media. By pre-gathering candidates into the pool, it can exchange and start sending connectivity checks from these candidates almost immediately upon receipt of a call. Note though that by holding on to these pre-gathered candidates, which will be kept alive as long as they may be needed, the application will consume resources on the STUN/TURN servers it is using. @@ -649,21 +674,22 @@ frame sizes it is capable of receiving. A receiver may have hard limits on what its video decoder can process, or it may wish to constrain what it receives due to application preferences, e.g. a specific size for the window in which the video will be displayed. Note that certain codecs support transmission of samples with aspect ratios other than 1.0 (i.e., non-square pixels). JSEP implementations will not transmit non-square pixels, but SHOULD receive and render such video with the correct aspect ratio. However, sample aspect ratio has no impact on the size negotiation - described below; all dimensions assume square pixels. + described below; all dimensions are measured in pixels, whether + square or not. 3.6.1. Creating an imageattr Attribute In order to determine the limits on what video resolution a receiver wants to receive, it will intersect its decoder hard limits with any mandatory constraints that have been applied to the associated MediaStreamTrack. If the decoder limits are unknown, e.g. when using a software decoder, the mandatory constraints are used directly. For the answerer, these mandatory constraints can be applied to the remote MediaStreamTracks that are created by a setRemoteDescription @@ -677,42 +703,44 @@ will be a new offer-answer to communicate them. If there are no known decoder limits or mandatory constraints, the "a=imageattr" attribute SHOULD be omitted. Otherwise, an "a=imageattr" attribute is created with "recv" direction, and the resulting resolution space formed by intersecting the decoder limits and constraints is used to specify its minimum and maximum x= and y= values. If the intersection is the null set, i.e., there are no resolutions that are permitted by both the decoder and - the mandatory constraints, this SHOULD be represented by x=0 and y=0 + the mandatory constraints, this MUST be represented by x=0 and y=0 values. The rules here express a single set of preferences, and therefore, the "a=imageattr" q= value is not important. It SHOULD be set to 1.0. The "a=imageattr" field is payload type specific. When all video codecs supported have the same capabilities, use of a single attribute, with the wildcard payload type (*), is RECOMMENDED. However, when the supported video codecs have differing capabilities, specific "a=imageattr" attributes MUST be inserted for each payload type. - As an example, consider a system with a HD-capable, multiformat video - decoder, where the application has constrained the received track to - at most 360p. In this case, the implementation would generate this + As an example, consider a system with a multiformat video decoder, + which is capable of decoding any resolution from 48x48 to 720p, and + where the application has constrained the received track to at most + 360p. In this case, the implementation would generate this attribute: - a=imageattr:* recv [x=[16:640],y=[16:360],q=1.0] + a=imageattr:* recv [x=[48:640],y=[48:360],q=1.0] + This declaration indicates that the receiver is capable of decoding - any image resolution from 16x16 up to 640x360 pixels. + any image resolution from 48x48 up to 640x360 pixels. 3.6.2. Interpreting an imageattr Attribute [RFC6236] defines "a=imageattr" to be an advisory field. This means that it does not absolutely constrain the video formats that the sender can use, but gives an indication of the preferred values. This specification prescribes more specific behavior. When a sender of a given MediaStreamTrack, which is producing video of a certain resolution, receives an "a=imageattr recv" attribute, it MUST check @@ -722,27 +750,29 @@ producing rotated video, the unrotated resolution MUST be used. This is required regardless of whether the receiver supports performing receive-side rotation (e.g., through CVO), as it significantly simplifies the matching logic. For the purposes of resolution negotiation, only size limits are considered. Any other values, e.g. picture or sample aspect ratio, MUST be ignored. When communicating with a non-JSEP endpoint, multiple relevant - "a=imageattr recv" attributes may be received. If this occurs, - attributes other than the one with the highest "q=" value MUST be - ignored. + "a=imageattr recv" attributes may be present in a received m= + section. If this occurs, attributes other than the one with the + highest "q=" value MUST be ignored. If multiple attributes have the + same "q=" value, those that appear after the first such attribute in + the m= section MUST be ignored. - If an "a=imageattr recv" attribute references a different video codec - than what has been selected for the MediaStreamTrack, it MUST be - ignored. + If an "a=imageattr recv" attribute references a different video + payload type than what has been selected for sending the + MediaStreamTrack, it MUST be ignored. If the original resolution matches the size limits in the attribute, the track MUST be transmitted untouched. If the original resolution exceeds the size limits in the attribute, the sender SHOULD apply downscaling to the output of the MediaStreamTrack in order to satisfy the limits. Downscaling MUST NOT change the track aspect ratio. If the original resolution is less than the size limits in the @@ -761,66 +791,67 @@ If the attribute includes a "sar=" (sample aspect ratio) value set to something other than "1.0", indicating the receiver wants to receive non-square pixels, this cannot be satisfied and the sender MUST NOT transmit the track. In the special case of receiving a maximum resolution of [0, 0], as described above, the sender MUST NOT transmit the track. 3.7. Simulcast - JSEP supports simulcast of a MediaStreamTrack, where multiple - encodings of the source media can be transmitted within the context - of a single m= section. The current JSEP API is designed to allow - applications to send simulcasted media but only to receive a single - encoding. This allows for multi-user scenarios where each sending - client sends multiple encodings to a server, which then, for each - receiving client, chooses the appropriate encoding to forward. + JSEP supports simulcast transmission of a MediaStreamTrack, where + multiple encodings of the source media can be transmitted within the + context of a single m= section. The current JSEP API is designed to + allow applications to send simulcasted media but only to receive a + single encoding. This allows for multi-user scenarios where each + sending client sends multiple encodings to a server, which then, for + each receiving client, chooses the appropriate encoding to forward. Applications request support for simulcast by configuring multiple - encodings on an RTPSender, which, upon generation of an offer or + encodings on an RtpSender, which, upon generation of an offer or answer, are indicated in SDP markings on the corresponding m= section, as described below. Receivers that understand simulcast and are willing to receive it will also include SDP markings to indicate their support, and JSEP endpoints will use these markings to - determine whether simulcast is permitted for a given RTPSender. If - simulcast support is not negotiated, the RTPSender will only use the + determine whether simulcast is permitted for a given RtpSender. If + simulcast support is not negotiated, the RtpSender will only use the first configured encoding. Note that the exact simulcast parameters are up to the sending application. While the aforementioned SDP markings are provided to ensure the remote side can receive and demux multiple simulcast encodings, the specific resolutions and bitrates to be used for each encoding are purely a send-side decision in JSEP. - JSEP currently does not provide an API to configure receipt of + JSEP currently does not provide a mechanism to configure receipt of simulcast. This means that if simulcast is offered by the remote endpoint, the answer generated by a JSEP endpoint will not indicate support for receipt of simulcast, and as such the remote endpoint - will only send a single encoding per m= section. In addition, when - the JSEP endpoint is the answerer, the permitted encodings for the - RTPSender must be consistent with the offer, but this information is - currently not surfaced through any API. This means that established - simulcast streams will continue to work through a received re-offer, - but setting up initial simulcast by way of a received offer requires - out-of-band signaling or SDP inspection. Future versions of this - specification may add additional APIs to provide this control. + will only send a single encoding per m= section. - When using JSEP to transmit multiple encodings from a RTPSender, the + In addition, JSEP does not provide a mechanism to handle an incoming + offer requesting simulcast from the JSEP endpoint. This means that + established simulcast streams will continue to work through a + received re-offer, but setting up initial simulcast by way of a + received offer requires out-of-band signaling or SDP inspection. + Future versions of this specification may add additional APIs to + provide direct control. + + When using JSEP to transmit multiple encodings from a RtpSender, the techniques from [I-D.ietf-mmusic-sdp-simulcast] and [I-D.ietf-mmusic-rid] are used. Specifically, when multiple - encodings have been configured for a RTPSender, the m= section for - the RTPSender will include an "a=simulcast" attribute, as defined in + encodings have been configured for a RtpSender, the m= section for + the RtpSender will include an "a=simulcast" attribute, as defined in [I-D.ietf-mmusic-sdp-simulcast], Section 6.2, with a "send" simulcast stream description that lists each desired encoding, and no "recv" simulcast stream description. The m= section will also include an - "a=rid" attribute for each encoding, as specfied in + "a=rid" attribute for each encoding, as specified in [I-D.ietf-mmusic-rid], Section 4; the use of RID identifiers allows the individual encodings to be disambiguated even though they are all part of the same m= section. 3.8. Interactions With Forking Some call signaling systems allow various types of forking where an SDP Offer may be provided to more than one device. For example, SIP [RFC3261] defines both a "Parallel Search" and "Sequential Search". Although these are primarily signaling level issues that are outside @@ -889,25 +920,25 @@ that it wishes to exchange media with. In JSEP, this offer used in the UPDATE would be formed by simply creating a new PeerConnection and making sure that the same local media streams have been added into this new PeerConnection. Then the new PeerConnection object would produce a SDP offer that could be used by the signaling to perform the UPDATE strategy discussed in [RFC3960]. As a result of sharing the media streams, the application will end up with N parallel PeerConnection sessions, each with a local and remote description and their own local and remote addresses. The media flow - from these sessions can be managed by specifying SDP direction - attributes in the descriptions, or the application can choose to play - out the media from all sessions mixed together. Of course, if the - application wants to only keep a single session, it can simply - terminate the sessions that it no longer needs. + from these sessions can be managed using setDirection (see + Section 4.2.3), or the application can choose to play out the media + from all sessions mixed together. Of course, if the application + wants to only keep a single session, it can simply terminate the + sessions that it no longer needs. 4. Interface This section details the basic operations that must be present to implement JSEP functionality. The actual API exposed in the W3C API may have somewhat different syntax, but should map easily to these concepts. 4.1. PeerConnection @@ -924,22 +955,22 @@ candidates (including any internal browser filtering) to the application, and only use those candidates for connectivity checks. The set of available policies is as follows: all: All candidates permitted by browser policy will be gathered and used. relay: All candidates except relay candidates will be filtered out. This obfuscates the location information that might be ascertained by the remote peer from the received candidates. Depending on how - the application deploys its relay servers, this could obfuscate - location to a metro or possibly even global level. + the application deploys and chooses relay servers, this could + obfuscate location to a metro or possibly even global level. The default ICE candidate policy MUST be set to "all" as this is generally the desired policy, and also typically reduces use of application TURN server resources significantly. If a size is specified for the ICE candidate pool, this indicates the number of ICE components to pre-gather candidates for. Because pre- gathering results in utilizing STUN/TURN server resources for potentially long periods of time, this must only occur upon application request, and therefore the default candidate pool size @@ -990,71 +1021,81 @@ be set to "balanced". The application can specify its preferred policy regarding use of RTP/RTCP multiplexing [RFC5761] using one of the following policies: negotiate: The browser will gather both RTP and RTCP candidates but also will offer "a=rtcp-mux", thus allowing for compatibility with either multiplexing or non-multiplexing endpoints. require: The browser will only gather RTP candidates. This halves - the number of candidates that the offerer needs to gather. When - acting as answerer, the implementation will reject any m= section - that does not contain an "a=rtcp-mux" attribute. + the number of candidates that the offerer needs to gather. + Applying a description with an m= section that does not contain an + "a=rtcp-mux" attribute will cause an error to be returned. The default multiplexing policy MUST be set to "require". Implementations MAY choose to reject attempts by the application to set the multiplexing policy to "negotiate". 4.1.2. addTrack The addTrack method adds a MediaStreamTrack to the PeerConnection, using the MediaStream argument to associate the track with other tracks in the same MediaStream, so that they can be added to the same "LS" group when creating an offer or answer. addTrack attempts to minimize the number of transceivers as follows: If the PeerConnection is in the "have-remote-offer" state, the track will be attached to the first compatible transceiver that was created by the most recent call to setRemoteDescription() and does not have a local track. Otherwise, a new transceiver will be created, as described in - Section 4.1.3. + Section 4.1.4. -4.1.3. addTransceiver +4.1.3. removeTrack - The addTransceiver method adds a new RTPTransceiver to the + The removeTrack method removes a MediaStreamTrack from the + PeerConnection, using the RtpSender argument to indicate which sender + should have its track removed. The sender's track is cleared, and + the sender stops sending. Future calls to createOffer will mark the + media description associated with the sender as recvonly (if + transceiver.currentDirection is sendrecv) or as inactive (if + transceiver.currentDirection is sendonly). + +4.1.4. addTransceiver + + The addTransceiver method adds a new RtpTransceiver to the PeerConnection. If a MediaStreamTrack argument is provided, then the transceiver will be configured with that media type and the track will be attached to the transceiver. Otherwise, the application MUST explicitly specify the type; this mode is useful for creating recvonly transceivers as well as for creating transceivers to which a track can be attached at some later point. At the time of creation, the application can also specify a transceiver direction attribute, a set of MediaStreams which the transceiver is associated with (allowing LS group assignments), and a set of encodings for the media (used for simulcast as described in Section 3.7). -4.1.4. createDataChannel +4.1.5. createDataChannel The createDataChannel method creates a new data channel and attaches it to the PeerConnection. If no data channel currently exists for this PeerConnection, then a new offer/answer exchange is required. All data channels on a given PeerConnection share the same SCTP/DTLS association and therefore the same m= section, so subsequent creation of data channels does not have any impact on the JSEP state. The createDataChannel method also includes a number of arguments which are used by the PeerConnection (e.g., maxPacketLifetime) but are not reflected in the SDP and do not affect the JSEP state. -4.1.5. createOffer +4.1.6. createOffer The createOffer method generates a blob of SDP that contains a [RFC3264] offer with the supported configurations for the session, including descriptions of the media added to this PeerConnection, the codec/RTP/RTCP options supported by this implementation, and any candidates that have been gathered by the ICE Agent. An options parameter may be supplied to provide additional control over the generated offer. This options parameter allows an application to trigger an ICE restart, for the purpose of reestablishing connectivity. @@ -1079,29 +1120,27 @@ are unaffected by the requested changes, the offer will only contain the parameters negotiated by the last offer-answer exchange. The exact handling of subsequent offer generation is detailed in Section 5.2.2. below. Session descriptions generated by createOffer must be immediately usable by setLocalDescription; if a system has limited resources (e.g. a finite number of decoders), createOffer should return an offer that reflects the current state of the system, so that setLocalDescription will succeed when it attempts to acquire those - resources. Because this method may need to inspect the system state - to determine the currently available resources, it may be implemented - as an async operation. + resources. Calling this method may do things such as generate new ICE credentials, but does not result in candidate gathering, or cause media to start or stop flowing. -4.1.6. createAnswer +4.1.7. createAnswer The createAnswer method generates a blob of SDP that contains a [RFC3264] SDP answer with the supported configuration for the session that is compatible with the parameters supplied in the most recent call to setRemoteDescription, which MUST have been called prior to calling createAnswer. Like createOffer, the returned blob contains descriptions of the media added to this PeerConnection, the codec/RTP/RTCP options negotiated for this session, and any candidates that have been gathered by the ICE Agent. An options parameter may be supplied to provide additional control over the @@ -1109,30 +1148,27 @@ As an answer, the generated SDP will contain a specific configuration that specifies how the media plane should be established; for each SDP line, the generation of the SDP must follow the process defined for generating an answer from the document that specifies the given SDP line. The exact handling of answer generation is detailed in Section 5.3. below. Session descriptions generated by createAnswer must be immediately usable by setLocalDescription; like createOffer, the returned - description should reflect the current state of the system. Because - this method may need to inspect the system state to determine the - currently available resources, it may need to be implemented as an - async operation. + description should reflect the current state of the system. Calling this method may do things such as generate new ICE credentials, but does not trigger candidate gathering or change media state. -4.1.7. SessionDescriptionType +4.1.8. SessionDescriptionType Session description objects (RTCSessionDescription) may be of type "offer", "pranswer", "answer" or "rollback". These types provide information as to how the description parameter should be parsed, and how the media state should be changed. "offer" indicates that a description should be parsed as an offer; said description may include many possible media configurations. A description used as an "offer" may be applied anytime the PeerConnection is in a stable state, or as an update to a previously @@ -1158,23 +1194,23 @@ provisional or final, and can change the type of the session description as needed. For example, in a serial forking scenario, an application may receive multiple "final" answers, one from each remote endpoint. The application could choose to accept the initial answers as provisional answers, and only apply an answer as final when it receives one that meets its criteria (e.g. a live user instead of voicemail). "rollback" is a special session description type implying that the state machine should be rolled back to the previous stable state, as - described in Section 4.1.7.2. The contents MUST be empty. + described in Section 4.1.8.2. The contents MUST be empty. -4.1.7.1. Use of Provisional Answers +4.1.8.1. Use of Provisional Answers Most web applications will not need to create answers using the "pranswer" type. While it is good practice to send an immediate response to an "offer", in order to warm up the session transport and prevent media clipping, the preferred handling for a web application would be to create and send an "inactive" final answer immediately after receiving the offer. Later, when the called user actually accepts the call, the application can create a new "sendrecv" offer to update the previous offer/answer pair and start the media flow. While this could also be done with an inactive "pranswer", followed @@ -1192,155 +1228,161 @@ two-way media. By the time the human has accepted the call and triggered the new offer, it is likely that the ICE and DTLS handshaking for all the channels will already have finished. Of course, some applications may not be able to perform this double offer-answer exchange, particularly ones that are attempting to gateway to legacy signaling protocols. In these cases, "pranswer" can still provide the application with a mechanism to warm up the transport. -4.1.7.2. Rollback +4.1.8.2. Rollback In certain situations it may be desirable to "undo" a change made to setLocalDescription or setRemoteDescription. Consider a case where a call is ongoing, and one side wants to change some of the session parameters; that side generates an updated offer and then calls setLocalDescription. However, the remote side, either before or after setRemoteDescription, decides it does not want to accept the new parameters, and sends a reject message back to the offerer. Now, the offerer, and possibly the answerer as well, need to return to a stable state and the previous local/remote description. To support this, we introduce the concept of "rollback". A rollback discards any proposed changes to the session, returning the state machine to the stable state, and setting the pending local - and/or remote description back to null. Any resources or candidates - that were allocated by the abandoned local description are discarded; - any media that is received will be processed according to the - previous local and remote descriptions. Rollback can only be used to - cancel proposed changes; there is no support for rolling back from a - stable state to a previous stable state. Note that this implies that - once the answerer has performed setLocalDescription with his answer, - this cannot be rolled back. + and/or remote description (see Section 4.1.12 and Section 4.1.14) to + null. Any resources or candidates that were allocated by the + abandoned local description are discarded; any media that is received + will be processed according to the previous local and remote + descriptions. Rollback can only be used to cancel proposed changes; + there is no support for rolling back from a stable state to a + previous stable state. Note that this implies that once the answerer + has performed setLocalDescription with his answer, this cannot be + rolled back. A rollback will disassociate any RtpTransceivers that were associated with m= sections by the application of the rolled-back session description (see Section 5.9 and Section 5.8). This means that some RtpTransceivers that were previously associated will no longer be associated with any m= section; in such cases, the value of the - RtpTransceiver's mid attribute MUST be set to null. RtpTransceivers - that were created by applying a remote offer that was subsequently - rolled back MUST be removed. However, a RtpTransceiver MUST NOT be - removed if the RtpTransceiver's RtpSender was activated by the - addTrack method. This is so that an application may call addTrack, - then call setRemoteDescription with an offer, then roll back that - offer, then call createOffer and have a m= section for the added - track appear in the generated offer. + RtpTransceiver's mid property MUST be set to null, and the mapping + between the transceiver and its m= section index MUST be discarded. + RtpTransceivers that were created by applying a remote offer that was + subsequently rolled back MUST be stopped and removed from the + PeerConnection. However, a RtpTransceiver MUST NOT be removed if a + track was attached to the RtpTransceiver via the addTrack method. + This is so that an application may call addTrack, then call + setRemoteDescription with an offer, then roll back that offer, then + call createOffer and have a m= section for the added track appear in + the generated offer. A rollback is performed by supplying a session description of type "rollback" with empty contents to either setLocalDescription or setRemoteDescription, depending on which was most recently used (i.e. if the new offer was supplied to setLocalDescription, the rollback should be done using setLocalDescription as well). -4.1.8. setLocalDescription +4.1.9. setLocalDescription The setLocalDescription method instructs the PeerConnection to apply the supplied session description as its local configuration. The type field indicates whether the description should be processed as an offer, provisional answer, or final answer; offers and answers are checked differently, using the various rules that exist for each SDP line. This API changes the local media state; among other things, it sets up local resources for receiving and decoding media. In order to successfully handle scenarios where the application wants to offer to change from one media format to a different, incompatible format, the PeerConnection must be able to simultaneously support use of both the - current and pending local descriptions (e.g. support codecs that - exist in both descriptions) until a final answer is received, at - which point the PeerConnection can fully adopt the pending local - description, or roll back to the current description if the remote - side denied the change. + current and pending local descriptions (e.g., support the codecs that + exist in either description). This dual processing begins when the + PeerConnection enters the have-local-offer state, and continues until + setRemoteDescription is called with either a final answer, at which + point the PeerConnection can fully adopt the pending local + description, or a rollback, which results in a revert to the current + local description. This API indirectly controls the candidate gathering process. When a local description is supplied, and the number of transports currently in use does not match the number of transports needed by the local description, the PeerConnection will create transports as needed and - begin gathering candidates for them. + begin gathering candidates for each transport, using ones from the + candidate pool if available. If setRemoteDescription was previously called with an offer, and setLocalDescription is called with an answer (provisional or final), - and the media directions are compatible, and media are available to + and the media directions are compatible, and media is available to send, this will result in the starting of media transmission. -4.1.9. setRemoteDescription +4.1.10. setRemoteDescription The setRemoteDescription method instructs the PeerConnection to apply the supplied session description as the desired remote configuration. As in setLocalDescription, the type field of the description indicates how it should be processed. This API changes the local media state; among other things, it sets up local resources for sending and encoding media. If setLocalDescription was previously called with an offer, and setRemoteDescription is called with an answer (provisional or final), - and the media directions are compatible, and media are available to + and the media directions are compatible, and media is available to send, this will result in the starting of media transmission. -4.1.10. currentLocalDescription +4.1.11. currentLocalDescription - The currentLocalDescription method returns a copy of the current - negotiated local description - i.e., the local description from the - last successful offer/answer exchange - in addition to any local + The currentLocalDescription method returns the current negotiated + local description - i.e., the local description from the last + successful offer/answer exchange - in addition to any local candidates that have been generated by the ICE Agent since the local description was set. A null object will be returned if an offer/answer exchange has not yet been completed. -4.1.11. pendingLocalDescription +4.1.12. pendingLocalDescription The pendingLocalDescription method returns a copy of the local description currently in negotiation - i.e., a local offer set without any corresponding remote answer - in addition to any local candidates that have been generated by the ICE Agent since the local description was set. A null object will be returned if the state of the PeerConnection is "stable" or "have-remote-offer". -4.1.12. currentRemoteDescription +4.1.13. currentRemoteDescription The currentRemoteDescription method returns a copy of the current negotiated remote description - i.e., the remote description from the last successful offer/answer exchange - in addition to any remote candidates that have been supplied via processIceMessage since the remote description was set. A null object will be returned if an offer/answer exchange has not yet been completed. -4.1.13. pendingRemoteDescription +4.1.14. pendingRemoteDescription The pendingRemoteDescription method returns a copy of the remote description currently in negotiation - i.e., a remote offer set without any corresponding local answer - in addition to any remote candidates that have been supplied via processIceMessage since the remote description was set. A null object will be returned if the state of the PeerConnection is "stable" or "have-local-offer". -4.1.14. canTrickleIceCandidates +4.1.15. canTrickleIceCandidates The canTrickleIceCandidates property indicates whether the remote side supports receiving trickled candidates. There are three potential values: null: No SDP has been received from the other side, so it is not known if it can handle trickle. This is the initial value before setRemoteDescription() is called. true: SDP has been received from the other side indicating that it @@ -1354,68 +1396,69 @@ needed for Trickle ICE. However, applications can use the canTrickleIceCandidates property to determine whether their peer can actually do Trickle ICE, i.e., whether it is safe to send an initial offer or answer followed later by candidates as they are gathered. As "true" is the only value that definitively indicates remote Trickle ICE support, an application which compares canTrickleIceCandidates against "true" will by default attempt Half Trickle on initial offers and Full Trickle on subsequent interactions with a Trickle ICE-compatible agent. -4.1.15. setConfiguration +4.1.16. setConfiguration The setConfiguration method allows the global configuration of the PeerConnection, which was initially set by constructor parameters, to be changed during the session. The effects of this method call depend on when it is invoked, and differ depending on which specific parameters are changed: o Any changes to the STUN/TURN servers to use affect the next gathering phase. If an ICE gathering phase has already started or completed, the 'needs-ice-restart' bit mentioned in Section 3.5.1 will be set. This will cause the next call to createOffer to generate new ICE credentials, for the purpose of forcing an ICE restart and kicking off a new gathering phase, in which the new servers will be used. If the ICE candidate pool has a nonzero - size, any existing candidates will be discarded, and new - candidates will be gathered from the new servers. + size, and a local description has not yet been applied, any + existing candidates will be discarded, and new candidates will be + gathered from the new servers. o Any change to the ICE candidate policy affects the next gathering phase. If an ICE gathering phase has already started or completed, the 'needs-ice-restart' bit will be set. Either way, changes to the policy have no effect on the candidate pool, because pooled candidates are not surfaced to the application until a gathering phase occurs, and so any necessary filtering can still be done on any pooled candidates. - o Any changes to the ICE candidate pool size take effect + o The ICE candidate pool size MUST NOT be changed after applying a + local description. If a local description has not yet been + applied, any changes to the ICE candidate pool size take effect immediately; if increased, additional candidates are pre-gathered; if decreased, the now-superfluous candidates are discarded. o The bundle and RTCP-multiplexing policies MUST NOT be changed after the construction of the PeerConnection. - This call may result in a change to the state of the ICE Agent, and - may result in a change to media state if it results in connectivity - being established. + This call may result in a change to the state of the ICE Agent. -4.1.16. addIceCandidate +4.1.17. addIceCandidate The addIceCandidate method provides a remote candidate to the ICE Agent, which, if parsed successfully, will be added to the current and/or pending remote description according to the rules defined for Trickle ICE. The pair of MID and ufrag is used to determine the m= section and ICE candidate generation to which the candidate belongs. - If the MID is not present, the m= line index is used to look up the - locally generated MID (see Section 5.9), which is used in place of a - supplied MID. If these values or the candidate string are invalid, - an error is generated. + If the MID is not present, the m= section index is used to look up + the locally generated MID (see Section 5.9), which is used in place + of a supplied MID. If these values or the candidate string are + invalid, an error is generated. The purpose of the ufrag is to resolve ambiguities when trickle ICE is in progress during an ICE restart. If the ufrag is absent, the candidate MUST be assumed to belong to the most recently applied remote description. Connectivity checks will be sent to the new candidate. This method can also be used to provide an end-of-candidates indication to the ICE Agent, as defined in [I-D.ietf-ice-trickle]). The MID and ufrag are used as described above to determine the m= @@ -1433,47 +1476,75 @@ 4.2. RtpTransceiver 4.2.1. stop The stop method stops an RtpTransceiver. This will cause future calls to createOffer to generate a zero port for the associated m= section. See below for more details. 4.2.2. stopped - The stopped method returns "true" if the transceiver has been + The stopped property indicates whether the transceiver has been stopped, either by a call to stopTransceiver or by applying an answer - that rejects the associated m= section, and "false" otherwise. + that rejects the associated m= section. In either of these cases, it + is set to "true", and otherwise will be set to "false". A stopped RtpTransceiver does not send any outgoing RTP or RTCP or process any incoming RTP or RTCP. It cannot be restarted. 4.2.3. setDirection The setDirection method sets the direction of a transceiver, which - affects the direction attribute of the associated m= section on - future calls to createOffer and createAnswer. + affects the direction property of the associated m= section on future + calls to createOffer and createAnswer. When creating offers, the transceiver direction is directly reflected in the output, even for reoffers. When creating answers, the transceiver direction is intersected with the offered direction, as explained in the Section 5.3 section below. -4.2.4. setCodecPreferences + Note that while setDirection sets the direction property of the + transceiver immediately (Section 4.2.4), this property does not + immediately affect whether the transceiver's RtpSender will send or + its RtpReceiver will receive. The direction in effect is represented + by the currentDirection property, which is only updated when an + answer is applied. + +4.2.4. direction + + The direction property indicates the last value passed into + setDirection. If setDirection has never been called, it is set to + the direction the transceiver was initialized with. + +4.2.5. currentDirection + + The currentDirection property indicates the last negotiated direction + for the transceiver's associated m= section. More specifically, it + indicates the [RFC3264] directional attribute of the associated m= + section in the last applied answer, with "send" and "recv" directions + reversed if it was a remote answer. For example, if the directional + attribute for the associated m= section in a remote answer is + "recvonly", currentDirection is set to "sendonly". + + If an answer that references this transceiver has not yet been + applied, or if the transceiver is stopped, currentDirection is set to + null. + +4.2.6. setCodecPreferences The setCodecPreferences method sets the codec preferences of a transceiver, which in turn affect the presence and order of codecs of the associated m= section on future calls to createOffer and createAnswer. Note that setCodecPreferences does not directly affect - which codec the implemtation decides to send. It only affects which - codecs the implementation indicates that it prefers to receive, via - the offer or answer. Even when a codec is excluded by + which codec the implementation decides to send. It only affects + which codecs the implementation indicates that it prefers to receive, + via the offer or answer. Even when a codec is excluded by setCodecPreferences, it still may be used to send until the next offer/answer exchange discards it. The codec preferences of an RtpTransceiver can cause codecs to be excluded by subsequent calls to createOffer and createAnswer, in which case the corresponding media formats in the associated m= section will be excluded. The codec preferences cannot add media formats that would otherwise not be present. This includes codecs that were not negotiated in a previous offer/answer exchange that included the transceiver. @@ -1500,95 +1571,92 @@ All implementations must support these behaviors, but may not use all of them if the remote side, which may not be a JSEP endpoint, does not support them. The second set of specifications is the "mandatory-to-use" set. The local JSEP endpoint and any remote endpoint must indicate support for these specifications in their session descriptions. 5.1.1. Implementation Requirements + Implementations of JSEP MUST conform to [I-D.ietf-rtcweb-rtp-usage]. This list of mandatory-to-implement specifications is derived from - the requirements outlined in [I-D.ietf-rtcweb-rtp-usage]. + the requirements outlined in that document and from + [I-D.ietf-rtcweb-security-arch]. R-1 [RFC4566] is the base SDP specification and MUST be implemented. R-2 [RFC5764] MUST be supported for signaling the UDP/TLS/RTP/SAVPF - [RFC5764], TCP/DTLS/RTP/SAVPF - [I-D.nandakumar-mmusic-proto-iana-registration], "UDP/DTLS/ - SCTP" [I-D.ietf-mmusic-sctp-sdp], and "TCP/DTLS/SCTP" + [RFC5764], TCP/DTLS/RTP/SAVPF [RFC7850], "UDP/DTLS/SCTP" + [I-D.ietf-mmusic-sctp-sdp], and "TCP/DTLS/SCTP" [I-D.ietf-mmusic-sctp-sdp] RTP profiles. R-3 [RFC5245] MUST be implemented for signaling the ICE credentials and candidate lines corresponding to each media stream. The ICE implementation MUST be a Full implementation, not a Lite implementation. R-4 [RFC5763] MUST be implemented to signal DTLS certificate fingerprints. - R-5 [RFC4568] MUST NOT be implemented to signal SDES SRTP keying - information. - - R-6 The [RFC5888] grouping framework MUST be implemented for - signaling grouping information, and MUST be used to identify m= - lines via the a=mid attribute. + R-5 [RFC5888] MUST be implemented for signaling grouping + information, and MUST be used to identify m= lines via the + a=mid attribute. - R-7 [I-D.ietf-mmusic-msid] MUST be supported, in order to signal + R-6 [I-D.ietf-mmusic-msid] MUST be supported, in order to signal associations between RTP objects and W3C MediaStreams and MediaStreamTracks in a standard way. - R-8 The bundle mechanism in + R-7 The bundle mechanism in [I-D.ietf-mmusic-sdp-bundle-negotiation] MUST be supported to signal the ability to multiplex RTP streams on a single UDP port, in order to avoid excessive use of port number resources. - R-9 The SDP attributes of "sendonly", "recvonly", "inactive", and + R-8 The SDP attributes of "sendonly", "recvonly", "inactive", and "sendrecv" from [RFC4566] MUST be implemented to signal information about media direction. - R-10 [RFC5576] MUST be implemented to signal RTP SSRC values and + R-9 [RFC5576] MUST be implemented to signal RTP SSRC values and grouping semantics. - R-11 [RFC4585] MUST be implemented to signal RTCP based feedback. + R-10 [RFC4585] MUST be implemented to signal RTCP based feedback. - R-12 [RFC5761] MUST be implemented to signal multiplexing of RTP and + R-11 [RFC5761] MUST be implemented to signal multiplexing of RTP and RTCP. - R-13 [RFC5506] MUST be implemented to signal reduced-size RTCP + R-12 [RFC5506] MUST be implemented to signal reduced-size RTCP messages. - R-14 [RFC4588] MUST be implemented to signal RTX payload type + R-13 [RFC4588] MUST be implemented to signal RTX payload type associations. - R-15 [RFC3556] with bandwidth modifiers MAY be supported for - specifying RTCP bandwidth as a fraction of the media bandwidth, - RTCP fraction allocated to the senders and setting maximum - media bit-rate boundaries. + R-14 [RFC3556] MUST be supported for control of RTCP bandwidth + limits. - R-16 TODO: any others? + The SDES SRTP keying mechanism from [RFC4568] MUST NOT be + implemented, as discussed in [I-D.ietf-rtcweb-security-arch]. As required by [RFC4566], Section 5.13, JSEP implementations MUST ignore unknown attribute (a=) lines. 5.1.2. Usage Requirements All session descriptions handled by JSEP endpoints, both local and remote, MUST indicate support for the following specifications. If any of these are absent, this omission MUST be treated as an error. - R-1 ICE, as specified in [RFC5245], MUST be used. Note that the + U-1 ICE, as specified in [RFC5245], MUST be used. Note that the remote endpoint may use a Lite implementation; implementations MUST properly handle remote endpoints which do ICE-Lite. - R-2 DTLS [RFC6347] or DTLS-SRTP [RFC5763], MUST be used, as + U-2 DTLS [RFC6347] or DTLS-SRTP [RFC5763], MUST be used, as appropriate for the media type, as specified in [I-D.ietf-rtcweb-security-arch] 5.1.3. Profile Names and Interoperability For media m= sections, JSEP endpoints MUST support both the "UDP/TLS/ RTP/SAVPF" and "TCP/DTLS/RTP/SAVPF" profiles and MUST indicate one of these two profiles for each media m= line they produce in an offer. For data m= sections, JSEP endpoints must support both the "UDP/DTLS/ SCTP" and "TCP/DTLS/SCTP" profiles and MUST indicate one of these two @@ -1614,21 +1682,21 @@ o Because DTLS-SRTP is REQUIRED, the choice of SAVP or AVP has no effect; support for DTLS-SRTP is determined by the presence of one or more "a=fingerprint" attribute. Note that lack of an "a=fingerprint" attribute will lead to negotiation failure. o The use of AVPF or AVP simply controls the timing rules used for RTCP feedback. If AVPF is provided, or an "a=rtcp-fb" attribute is present, assume AVPF timing, i.e., a default value of "trr- int=0". Otherwise, assume that AVPF is being used in an AVP - compatible mode and use AVP timing, i.e., "trr-int=4". + compatible mode and use a value of "trr-int=4000". o For data m= sections, JSEP endpoints MUST support receiving the "UDP/ DTLS/SCTP", "TCP/DTLS/SCTP", or "DTLS/SCTP" (for backwards compatibility) profiles. Note that re-offers by JSEP endpoints MUST use the correct profile strings even if the initial offer/answer exchange used an (incorrect) older profile string. 5.2. Constructing an Offer @@ -1663,108 +1731,116 @@ a random number for is sufficient to accomplish this. o The third SDP line MUST be a "s=" line, as specified in [RFC4566], Section 5.3; to match the "o=" line, a single dash SHOULD be used as the session name, e.g. "s=-". Note that this differs from the advice in [RFC4566] which proposes a single space, but as both "o=" and "s=" are meaningless, having the same meaningless value seems clearer. o Session Information ("i="), URI ("u="), Email Address ("e="), - Phone Number ("p="), Bandwidth ("b="), Repeat Times ("r="), and - Time Zones ("z=") lines are not useful in this context and SHOULD - NOT be included. + Phone Number ("p="), Repeat Times ("r="), and Time Zones ("z=") + lines are not useful in this context and SHOULD NOT be included. o Encryption Keys ("k=") lines do not provide sufficient security and MUST NOT be included. o A "t=" line MUST be added, as specified in [RFC4566], Section 5.9; both and SHOULD be set to zero, e.g. "t=0 0". o An "a=ice-options" line with the "trickle" option MUST be added, as specified in [I-D.ietf-ice-trickle], Section 4. The next step is to generate m= sections, as specified in [RFC4566] Section 5.14. An m= section is generated for each RtpTransceiver - that has been added to the PeerConnection. This is done in the order - that their associated RtpTransceivers were added to the - PeerConnection and excludes RtpTransceivers that are stopped and not - associated with an m= section (either due to an m= section being - recycled or an RtpTransceiver having been stopped before being - associated with an m= section) . + that has been added to the PeerConnection, excluding any stopped + RtpTransceivers. This is done in the order the RtpTransceivers were + added to the PeerConnection. + + For each m= section generated for an RtpTransceiver, establish a + mapping between the transceiver and the index of the generated m= + section. Each m= section, provided it is not marked as bundle-only, MUST generate a unique set of ICE credentials and gather its own unique set of ICE candidates. Bundle-only m= sections MUST NOT contain any ICE credentials and MUST NOT gather any candidates. For DTLS, all m= sections MUST use all the certificate(s) that have been specified for the PeerConnection; as a result, they MUST all have the same [I-D.ietf-mmusic-4572-update] fingerprint value(s), or these value(s) MUST be session-level attributes. Each m= section should be generated as specified in [RFC4566], Section 5.14. For the m= line itself, the following rules MUST be followed: o The port value is set to the port of the default ICE candidate for - this m= section, but given that no candidates have yet been - gathered, the "dummy" port value of 9 (Discard) MUST be used, as - indicated in [I-D.ietf-ice-trickle], Section 5.1. + this m= section, but given that no candidates are available yet, + the "dummy" port value of 9 (Discard) MUST be used, as indicated + in [I-D.ietf-ice-trickle], Section 5.1. o To properly indicate use of DTLS, the field MUST be set to - "UDP/TLS/RTP/SAVPF", as specified in [RFC5764], Section 8, if the - default candidate uses UDP transport, or "TCP/DTLS/RTP/SAVPF", as - specified in [I-D.nandakumar-mmusic-proto-iana-registration] if - the default candidate uses TCP transport. + "UDP/TLS/RTP/SAVPF", as specified in [RFC5764], Section 8. o If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, and MUST exclude any codecs not present in the codec preferences. + o The media formats in the answer MAY include codecs present in the + offer that were discarded in a previous offer/answer exchange. + This is necessary for compatibility with third- party call control + and SIP use cases. + o Unless excluded by the above restrictions, the media formats MUST include the mandatory audio/video codecs as specified in [I-D.ietf-rtcweb-audio](see Section 3) and [I-D.ietf-rtcweb-video](see Section 5). The m= line MUST be followed immediately by a "c=" line, as specified - in [RFC4566], Section 5.7. Again, as no candidates have yet been - gathered, the "c=" line must contain the "dummy" value "IN IP4 - 0.0.0.0", as defined in [I-D.ietf-ice-trickle], Section 5.1. + in [RFC4566], Section 5.7. Again, as no candidates are available + yet, the "c=" line must contain the "dummy" value "IN IP4 0.0.0.0", + as defined in [I-D.ietf-ice-trickle], Section 5.1. [I-D.ietf-mmusic-sdp-mux-attributes] groups SDP attributes into different categories. To avoid unnecessary duplication when bundling, Section 8.1 of [I-D.ietf-mmusic-sdp-bundle-negotiation] specifies that attributes of category IDENTICAL or TRANSPORT should not be repeated in bundled m= sections. The following attributes, which are of a category other than IDENTICAL or TRANSPORT, MUST be included in each m= section: - o An "a=mid" line, as specified in [RFC5888], Section 4. When - generating mid values, it is RECOMMENDED that the values be 3 - bytes or less, to allow them to efficiently fit into the RTP - header extension defined in - [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 11. + o An "a=mid" line, as specified in [RFC5888], Section 4. All MID + values MUST be generated in a fashion that does not leak user + information, e.g., randomly or using a per-PeerConnection counter, + and SHOULD be 3 bytes or less, to allow them to efficiently fit + into the RTP header extension defined in + [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 14. Note that + this does not set the RtpTransceiver mid property, as that only + occurs when the description is applied. The generated MID value + can be considered a "proposed" MID at this point. o A direction attribute which is the same as that of the associated transceiver. o For each media format on the m= line, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6, and [RFC3264], Section 5.1. - o If this m= section is for media with configurable frame sizes, - e.g. audio, an "a=maxptime" line, indicating the smallest of the - maximum supported frame sizes out of all codecs included above, as - specified in [RFC4566], Section 6. + o If this m= section is for media with configurable durations of + media per packet, e.g., audio, an "a=maxptime" line, indicating + the maximum amount of media, specified in milliseconds, that can + be encapsulated in each packet, as specified in [RFC4566], + Section 6. This value is set to the smallest of the maximum + duration values across all the codecs included in the m= section. o If this m= section is for video media, and there are known limitations on the size of images which can be decoded, an "a=imageattr" line, as specified in Section 3.6. o For each primary codec where RTP retransmission should be used, a corresponding "a=rtpmap" line indicating "rtx" with the clock rate of the primary codec and an "a=fmtp" line that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. @@ -1797,24 +1873,25 @@ * An "a=msid" line, as specified in [I-D.ietf-mmusic-msid], Section 2. o If the RtpTransceiver has a sendrecv or sendonly direction, and the application has specified RID values or has specified more than one encoding in the RtpSenders's parameters, an "a=rid" line for each encoding specified. The "a=rid" line is specified in [I-D.ietf-mmusic-rid], and its direction MUST be "send". If the application has chosen a RID value, it MUST be used as the rid- identifier; otherwise a RID value MUST be generated by the - implementation. When generating RID values, it is RECOMMENDED - that the values be 3 bytes or less, to allow them to efficiently - fit into the RTP header extension defined in - [I-D.ietf-avtext-rid], Section 11. If no encodings have been + implementation. RID values MUST be generated in a fashion that + does not leak user information, e.g., randomly or using a per- + PeerConnection counter, and SHOULD be 3 bytes or less, to allow + them to efficiently fit into the RTP header extension defined in + [I-D.ietf-avtext-rid], Section 3. If no encodings have been specified, or only one encoding is specified but without a RID value, then no "a=rid" lines are generated. o If the RtpTransceiver has a sendrecv or sendonly direction and more than one "a=rid" line has been generated, an "a=simulcast" line, with direction "send", as defined in [I-D.ietf-mmusic-sdp-simulcast], Section 6.2. The list of RIDs MUST include all of the RID identifiers used in the "a=rid" lines for this m= section. @@ -1836,31 +1913,29 @@ clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the offer MUST be "actpass". o An "a=dtls-id" line, as specified in [I-D.ietf-mmusic-dtls-sdp] Section 5.2. o An "a=rtcp" line, as specified in [RFC3605], Section 2.1, containing the dummy value "9 IN IP4 0.0.0.0", because no candidates have yet been gathered. - o An "a=rtcp-mux" line, as specified in [RFC5761], Section 5.1.1. + o An "a=rtcp-mux" line, as specified in [RFC5761], Section 5.1.3. o An "a=rtcp-rsize" line, as specified in [RFC5506], Section 5. Lastly, if a data channel has been created, a m= section MUST be generated for data. The field MUST be set to "application" - and the field MUST be set to "UDP/DTLS/SCTP" if the default - candidate uses UDP transport, or "TCP/DTLS/SCTP" if the default - candidate uses TCP transport [I-D.ietf-mmusic-sctp-sdp]. The "fmt" - value MUST be set to "webrtc-datachannel" as specified in - [I-D.ietf-mmusic-sctp-sdp], Section 4.1. + and the field MUST be set to "UDP/DTLS/SCTP" + [I-D.ietf-mmusic-sctp-sdp]. The "fmt" value MUST be set to "webrtc- + datachannel" as specified in [I-D.ietf-mmusic-sctp-sdp], Section 4.1. Within the data m= section, the "a=mid", "a=ice-ufrag", "a=ice-pwd", "a=fingerprint", "a=dtls-id", and "a=setup" lines MUST be included as mentioned above, along with an "a=fmtp:webrtc-datachannel" line and an "a=sctp-port" line referencing the SCTP port number as defined in [I-D.ietf-mmusic-sctp-sdp], Section 4.1. Once all m= sections have been generated, a session-level "a=group" attribute MUST be added as specified in [RFC5888]. This attribute MUST have semantics "bundle", and MUST include the mid identifiers of @@ -1909,23 +1985,24 @@ for generating an initial offer should be followed, subject to the following restriction: o The fields of the "o=" line MUST stay the same except for the field, which MUST increment by one on each call to createOffer if the offer might differ from the output of the previous call to createOffer; implementations MAY opt to increment on every call. The value of the generated is independent of the of the current local description; in particular, in the case where the - current version is N, an offer is created with version N+1, and - then that offer is rolled back so that the current version is - again N, the next generated offer will still have version N+2. + current version is N, an offer is created and applied with version + N+1, and then that offer is rolled back so that the current + version is again N, the next generated offer will still have + version N+2. Note that if the application creates an offer by reading currentLocalDescription instead of calling createOffer, the returned SDP may be different than when setLocalDescription was originally called, due to the addition of gathered ICE candidates, but the will not have changed. There are no known scenarios in which this causes problems, but if this is a concern, the solution is simply to use createOffer to ensure a unique . @@ -1972,21 +2049,21 @@ o Each "a=ice-ufrag" and "a=ice-pwd" line MUST stay the same, unless the ICE configuration has changed (either changes to the supported STUN/TURN servers, or the ICE candidate policy), or the "IceRestart" option ( Section 5.2.3.1 was specified. If the m= section is bundled into another m= section, it still MUST NOT contain any ICE credentials. o If the m= section is not bundled into another m= section, an "a=rtcp" attribute line MUST be added with of the default RTCP - candidate, as indicated in [RFC5761], section 5.1.3. + candidate, as indicated in [RFC5761], Section 5.1.3. o If the m= section is not bundled into another m= section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an "a=candidate" line MUST be added, as defined in [RFC5245], Section 4.3., paragraph 3. If candidate gathering for the section has completed, an "a=end-of-candidates" attribute MUST be added, as described in [I-D.ietf-ice-trickle], Section 9.3. If the m= section is bundled into another m= section, both "a=candidate" and "a=end-of-candidates" MUST be omitted. @@ -2002,37 +2079,43 @@ o If any RtpTransceiver has been stopped, the port MUST be set to zero and the "a=msid" line MUST be removed. o If any RtpTransceiver has been added, and there exists a m= section with a zero port in the current local description or the current remote description, that m= section MUST be recycled by generating a m= section for the added RtpTransceiver as if the m= section were being added to session description, except that instead of adding it, the generated m= section replaces the m= - section with a zero port. + section with a zero port. The new m= section MUST contain a new + MID. If the initial offer was applied using setLocalDescription, and an answer from the remote side has been applied using setRemoteDescription, meaning the PeerConnection is in the "remote- pranswer" or "stable" states, an offer is generated based on the negotiated session descriptions by following the steps mentioned for the "local-offer" state above. In addition, for each non-recycled, non-rejected m= section in the new offer, the following adjustments are made based on the contents - of the corresponding m= section in the current remote description: + of the corresponding m= section in the current remote description, if + any: o The m= line and corresponding "a=rtpmap" and "a=fmtp" lines MUST only include codecs present in the most recent answer which have not been excluded by the codec preferences of the associated - transceiver. + transceiver. Note that non-JSEP endpoints are not subject to + these restrictions, and might offer media formats that were not + present in the most recent answer, as specified in [RFC3264], + Section 8. Therefore, JSEP endpoints MUST be prepared to receive + such offers. o The media formats on the m= line MUST be generated in the same order as in the current local description. o The RTP header extensions MUST only include those that are present in the most recent answer. o The RTCP feedback extensions MUST only include those that are present in the most recent answer. @@ -2170,49 +2253,50 @@ o No supported codec is present in the offer. o The bundle policy is "max-bundle", and this is not the first m= section or in the same bundle group as the first m= section. o The bundle policy is "balanced", and this is not the first m= section for this media type or in the same bundle group as the first m= section for this media type. - o The RTP/RTCP multiplexing policy is "require" and the m= section - doesn't contain an "a=rtcp-mux" attribute. - Otherwise, each m= section in the answer should then be generated as specified in [RFC3264], Section 6.1. For the m= line itself, the following rules must be followed: o The port value would normally be set to the port of the default ICE candidate for this m= section, but given that no candidates - have yet been gathered, the "dummy" port value of 9 (Discard) MUST - be used, as indicated in [I-D.ietf-ice-trickle], Section 5.1. + are available yet, the "dummy" port value of 9 (Discard) MUST be + used, as indicated in [I-D.ietf-ice-trickle], Section 5.1. o The field MUST be set to exactly match the field for the corresponding m= line in the offer. o If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, and MUST exclude any codecs not present in the codec preferences or - not present in the offer. + not present in the offer. Note that non-JSEP endpoints are not + subject to this restriction, and might add media formats in the + answer that are not present in the offer, as specified in + [RFC3264], Section 6.1. Therefore, JSEP endpoints MUST be + prepared to receive such answers. o Unless excluded by the above restrictions, the media formats MUST include the mandatory audio/video codecs as specified in [I-D.ietf-rtcweb-audio](see Section 3) and [I-D.ietf-rtcweb-video](see Section 5). The m= line MUST be followed immediately by a "c=" line, as specified - in [RFC4566], Section 5.7. Again, as no candidates have yet been - gathered, the "c=" line must contain the "dummy" value "IN IP4 - 0.0.0.0", as defined in [I-D.ietf-ice-trickle], Section 5.1. + in [RFC4566], Section 5.7. Again, as no candidates are available + yet, the "c=" line must contain the "dummy" value "IN IP4 0.0.0.0", + as defined in [I-D.ietf-ice-trickle], Section 5.1. If the offer supports bundle, all m= sections to be bundled must use the same ICE credentials and candidates; all m= sections not being bundled must use unique ICE credentials and candidates. Each m= section MUST contain the following attributes (which are of attribute types other than IDENTICAL and TRANSPORT): o If and only if present in the offer, an "a=mid" line, as specified in [RFC5888], Section 9.1. The "mid" value MUST match that specified in the offer. @@ -2221,24 +2305,23 @@ the offered direction specified in [RFC3264], Section 6.1, and then intersecting with the direction of the associated RtpTransceiver. For example, in the case where an m= section is offered as "sendonly", and the local transceiver is set to "sendrecv", the result in the answer is a "recvonly" direction. o For each media format on the m= line, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6, and [RFC3264], Section 6.1. - o If this m= section is for media with configurable frame sizes, - e.g. audio, an "a=maxptime" line, indicating the smallest of the - maximum supported frame sizes out of all codecs included above, as - specified in [RFC4566], Section 6. + o If this m= section is for media with configurable durations of + media per packet, e.g., audio, an "a=maxptime" line, as described + in Section 5.2. o If this m= section is for video media, and there are known limitations on the size of images which can be decoded, an "a=imageattr" line, as specified in Section 3.6. o If "rtx" is present in the offer, for each primary codec where RTP retransmission should be used, a corresponding "a=rtpmap" line indicating "rtx" with the clock rate of the primary codec and an "a=fmtp" line that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. @@ -2275,29 +2358,27 @@ Section 15.4. o An "a=fingerprint" line for each of the endpoint's certificates, as specified in [RFC4572], Section 5; the digest algorithm used for the fingerprint MUST match that used in the certificate signature. o An "a=setup" line, as specified in [RFC4145], Section 4, and clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the answer MUST be "active" or "passive"; the - "active" role is RECOMMENDED. The role value MUST be consistent - with the existing DTLS connection, if one exists and is being - continued. + "active" role is RECOMMENDED. o An "a=dtls-id" line, as specified in [I-D.ietf-mmusic-dtls-sdp] Section 5.3. o If present in the offer, an "a=rtcp-mux" line, as specified in - [RFC5761], Section 5.1.1. Otherwise, an "a=rtcp" line, as + [RFC5761], Section 5.1.3. Otherwise, an "a=rtcp" line, as specified in [RFC3605], Section 2.1, containing the dummy value "9 IN IP4 0.0.0.0" (because no candidates have yet been gathered). o If present in the offer, an "a=rtcp-rsize" line, as specified in [RFC5506], Section 5. If a data channel m= section has been offered, a m= section MUST also be generated for data. The field MUST be set to "application" and the and "fmt" fields MUST be set to exactly match the fields in the offer. @@ -2354,20 +2435,24 @@ o The media formats on the m= line MUST be generated in the same order as in the current local description. o Each "a=ice-ufrag" and "a=ice-pwd" line MUST stay the same, unless the m= section is restarting, in which case new ICE credentials must be created as specified in [RFC5245], Section 9.2.1.1. If the m= section is bundled into another m= section, it still MUST NOT contain any ICE credentials. + o Each "a=setup" line MUST use an "active" or "passive" role value + consistent with the existing DTLS association, if the association + is being continued by the offerer. + o If the m= section is not bundled into another m= section and RTCP multiplexing is not active, an "a=rtcp" attribute line MUST be filled in with the port and address of the default RTCP candidate. If no RTCP candidates have yet been gathered, dummy values MUST be used, as described in the initial answer section above. o If the m= section is not bundled into another m= section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an "a=candidate" line MUST be added, as defined in [RFC5245], Section 4.3., paragraph 3. If candidate @@ -2379,42 +2464,42 @@ o For RtpTransceivers that are not stopped, the "a=msid" line MUST stay the same. 5.3.3. Options Handling The createAnswer method takes as a parameter an RTCAnswerOptions object. The set of parameters for RTCAnswerOptions is different than those supported in RTCOfferOptions; the IceRestart option is unnecessary, as ICE credentials will automatically be changed for all - m= lines where the offerer chose to perform ICE restart. + m= sections where the offerer chose to perform ICE restart. The following options are supported in RTCAnswerOptions. 5.3.3.1. VoiceActivityDetection Silence suppression in the answer is handled as described in Section 5.2.3.2, with one exception: if support for silence suppression was not indicated in the offer, the VoiceActivityDetection parameter has no effect, and the answer should be generated as if VoiceActivityDetection was set to false. This is done on a per-codec basis (e.g., if the offerer somehow offered support for CN but set "usedtx=0" for Opus, setting VoiceActivityDetection to true would result in an answer with CN codecs and "usedtx=0"). 5.4. Modifying an Offer or Answer The SDP returned from createOffer or createAnswer MUST NOT be changed before passing it to setLocalDescription. If precise control over - the SDP is needed, the aformentioned createOffer/createAnswer options - or RTPSender APIs MUST be used. + the SDP is needed, the aforementioned createOffer/createAnswer + options or RtpTransceiver APIs MUST be used. Note that the application MAY modify the SDP to reduce the capabilities in the offer it sends to the far side (post- setLocalDescription) or the offer that it installs from the far side (pre-setRemoteDescription), as long as it remains a valid SDP offer and specifies a subset of what was in the original offer. This is safe because the answer is not permitted to expand capabilities, and therefore will just respond to what is present in the offer. The application SHOULD NOT modify the SDP in the answer it transmits, @@ -2642,21 +2727,21 @@ o Any "a=ssrc" or "a=ssrc-group" attributes MUST be parsed as specified in [RFC5576], Sections 4.1-4.2, and their values stored. o Any "a=extmap" attributes MUST be parsed as specified in [RFC5285], Section 5, and their values stored. o Any "a=rtcp-fb" attributes MUST be parsed as specified in [RFC4585], Section 4.2., and their values stored. o If present, a single "a=rtcp-mux" attribute MUST be parsed as - specified in [RFC5761], Section 5.1.1, and its presence or absence + specified in [RFC5761], Section 5.1.3, and its presence or absence flagged and stored. o If present, a single "a=rtcp-mux-only" attribute MUST be parsed as specified in [I-D.ietf-mmusic-mux-exclusive], Section 3, and its presence or absence flagged and stored. o If present, a single "a=rtcp-rsize" attribute MUST be parsed as specified in [RFC5506], Section 5, and its presence or absence flagged and stored. @@ -2724,84 +2809,94 @@ * DTLS fingerprint values, where at least one fingerprint MUST be present. o All RID values referenced in an "a=simulcast" line MUST exist as "a=rid" lines. o Each m= section is also checked to ensure prohibited features are not used. If this is a local description, the "ice-lite" attribute MUST NOT be specified. + o If the RTP/RTCP multiplexing policy is "require", each m= section + MUST contain an "a=rtcp-mux" attribute. + If this session description is of type "pranswer" or "answer", the following additional checks are applied: o The session description must follow the rules defined in [RFC3264], Section 6, including the requirement that the number of m= sections MUST exactly match the number of m= sections in the associated offer. o For each m= section, the media type and protocol values MUST exactly match the media type and protocol values in the corresponding m= section in the associated offer. + If any of the preceding checks failed, processing MUST stop and an + error MUST be returned. + 5.8. Applying a Local Description The following steps are performed at the media engine level to apply a local description. - First, the parsed parameters are checked to ensure that they have not - been altered after their generation in createOffer/createAnswer, as - discussed in Section 5.4; otherwise, processing MUST stop and an - error MUST be returned. + First, the parsed parameters are checked to ensure that they are + identical to those generated in the last call to createOffer/ + createAnswer, and thus have not been altered, as discussed in + Section 5.4; otherwise, processing MUST stop and an error MUST be + returned. Next, media sections are processed. For each media section, the following steps MUST be performed; if any parameters are out of bounds, or cannot be applied, processing MUST stop and an error MUST be returned. o If this media section is new, begin gathering candidates for it, as defined in [RFC5245], Section 4.1.1, unless it has been marked as bundle-only. o Or, if the ICE ufrag and password values have changed, and it has not been marked as bundle-only, trigger the ICE Agent to start an ICE restart, and begin gathering new candidates for the media section as described in [RFC5245], Section 9.1.1.1. If this description is an answer, also start checks on that media section as defined in [RFC5245], Section 9.3.1.1. o If the media section proto value indicates use of RTP: * If there is no RtpTransceiver associated with this m= section - (which should only happen when applying an offer), find one and + (which will only happen when applying an offer), find one and associate it with this m= section according to the following steps: - + Find the RtpTransceiver that corresponds to the m= section - with the same MID in the created offer. + + Find the RtpTransceiver that corresponds to this m= section, + using the mapping between transceivers and m= section + indices established when creating the offer. - + Set the value of the RtpTransceiver's mid attribute to the + + Set the value of this RtpTransceiver's mid property to the MID of the m= section. * If RTCP mux is indicated, prepare to demux RTP and RTCP from the RTP ICE component, as specified in [RFC5761], - Section 5.1.1. If RTCP mux is not indicated, but was indicated - in a previous description, this MUST result in an error. + Section 5.1.3. If RTCP mux is not indicated, but was + previously negotiated, i.e., the RTCP ICE component no longer + exists, this MUST result in an error. * For each specified RTP header extension, establish a mapping between the extension ID and URI, as described in section 6 of [RFC5285]. If any indicated RTP header extension is not supported, this MUST result in an error. * If the MID header extension is supported, prepare to demux RTP - data intended for this media section based on the MID header - extension, as described in [I-D.ietf-mmusic-msid], Section 3.2. + streams intended for this media section based on the MID header + extension, as described in + [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 14. * For each specified media format, establish a mapping between the payload type and the actual media format, as described in [RFC3264], Section 6.1. If any indicated media format is not supported, this MUST result in an error. * For each specified "rtx" media format, establish a mapping between the RTX payload type and its associated primary payload type, as described in [RFC4588], Sections 8.6 and 8.7. If any referenced primary payload types are not present, this MUST @@ -2821,24 +2916,24 @@ The following steps are performed at the media engine level to apply a remote description. The following steps MUST be performed for attributes at the session level; if any parameters are out of bounds, or cannot be applied, processing MUST stop and an error MUST be returned. o For any specified "CT" bandwidth value, set this as the limit for the maximum total bitrate for all m= sections, as specified in - Section 5.8 of [RFC4566]. The implementation can decide how to - allocate the available bandwidth between m= sections to - simultaneously meet any limits on individual m= sections, as well - as this overall session limit. + Section 5.8 of [RFC4566]. Within this overall limit, the + implementation can dynamically decide how to best allocate the + available bandwidth between m= sections, respecting any specific + limits that have been specified for individual m= sections. o For any specified "RR" or "RS" bandwidth values, handle as specified in [RFC3556], Section 2. o Any "AS" bandwidth value MUST be ignored, as the meaning of this construct at the session level is not well defined. For each media section, the following steps MUST be performed; if any parameters are out of bounds, or cannot be applied, processing MUST stop and an error MUST be returned. @@ -2862,44 +2957,47 @@ connectivity checks with the appropriate credentials. o If an "a=end-of-candidates" attribute is present, process the end- of-candidates indication as described in [I-D.ietf-ice-trickle] Section 11. o If the media section proto value indicates use of RTP: * If the m= section is being recycled (see Section 5.2.2), dissociate the currently associated RtpTransceiver by setting - its mid attribute to null. + its mid property to null, and discard the mapping between the + transceiver and its m= section index. * If the m= section is not associated with any RtpTransceiver (possibly because it was dissociated in the previous step), either find an RtpTransceiver or create one according to the following steps: + If the m= section is sendrecv or recvonly, and there are RtpTransceivers of the same type that were added to the PeerConnection by addTrack and are not associated with any m= section and are not stopped, find the first (according to the canonical order described in Section 5.2.1) such RtpTransceiver. + If no RtpTransceiver was found in the previous step, create one with a recvonly direction. + Associate the found or created RtpTransceiver with the m= section by setting the value of the RtpTransceiver's mid - attribute to the MID of the m= section. If the m= section - does not include a MID (i.e., the remote side does not - support the MID extension), generate a value for the - RtpTransceiver mid attribute, following the guidance for - "a=mid" mentioned in Section 5.2.1. + property to the MID of the m= section, and establish a + mapping between the transceiver and the index of the m= + section. If the m= section does not include a MID (i.e., + the remote endpoint does not support the MID extension), + generate a value for the RtpTransceiver mid property, + following the guidance for "a=mid" mentioned in + Section 5.2.1. * For each specified media format that is also supported by the local implementation, establish a mapping between the specified payload type and the media format, as described in [RFC3264], Section 6.1. Specifically, this means that the implementation records the payload type to be used in outgoing RTP packets when sending each specified media format, as well as the relative preference for each format that is indicated in their ordering. If any indicated media format is not supported by the local implementation, it MUST be ignored. @@ -2931,22 +3029,22 @@ constraint on the maximum RTP bitrate to be used when sending media, as specified in [RFC3890]. If a "TIAS" value is not present, but an "AS" value is specified, generate a "TIAS" value using this formula: TIAS = AS * 1000 * 0.95 - 50 * 40 * 8 The 50 is based on 50 packets per second, the 40 is based on an estimate of total header size, the 1000 changes the unit from kbps to bps (as required by TIAS), and the 0.95 is to allocate - 5% to RTCP. If more accurate control of bandwidth is needed, - "TIAS" should be used instead of "AS". + 5% to RTCP. "TIAS" is used in preference to "AS" because it + provides more accurate control of bandwidth. * For any "RR" or "RS" bandwidth values, handle as specified in [RFC3556], Section 2. * Any specified "CT" bandwidth value MUST be ignored, as the meaning of this construct at the media level is not well defined. * If the media section is of type audio: @@ -2975,47 +3073,48 @@ 5.10. Applying an Answer In addition to the steps mentioned above for processing a local or remote description, the following steps are performed when processing a description of type "pranswer" or "answer". For each media section, the following steps MUST be performed: o If the media section has been rejected (i.e. port is set to zero in the answer), stop any reception or transmission of media for - this section, and discard any associated ICE components, as + this section, and, unless a non-rejected media section is bundled + with this media section, discard any associated ICE components, as described in Section 9.2.1.3 of [RFC5245]. o If the remote DTLS fingerprint has been changed or the dtls-id has changed, tear down the DTLS connection. If a DTLS connection needs to be torn down but the answer does not indicate an ICE restart, an error MUST be generated. If an ICE restart is performed without a change in dtls-id or fingerprint, then the same DTLS connection is continued over the new ICE channel. o If no valid DTLS connection exists, prepare to start a DTLS connection, using the specified roles and fingerprints, on any underlying ICE components, once they are active. o If the media section proto value indicates use of RTP: * If the media section references any media formats, RTP header extensions, or RTCP feedback mechanisms that were not present in the corresponding media section in the offer, this indicates a negotiation problem and MUST result in an error. - * If the media section has RTCP mux enabled, discard any RTCP - component, and begin or continue muxing RTCP over the RTP - component, as specified in [RFC5761], Section 5.1.3. - Otherwise, prepare to transmit RTCP over the RTCP component; if - no RTCP component exists, because RTCP mux was previously - enabled, this MUST result in an error. + * If the media section has RTCP mux enabled, discard the RTCP ICE + component, if one exists, and begin or continue muxing RTCP + over the RTP ICE component, as specified in [RFC5761], + Section 5.1.3. Otherwise, prepare to transmit RTCP over the + RTCP ICE component; if no RTCP ICE component exists, because + RTCP mux was previously enabled, this MUST result in an error. * If the media section has reduced-size RTCP enabled, configure the RTCP transmission for this media section to use reduced- size RTCP, as specified in [RFC5506]. * If the directional attribute in the answer is of type "sendrecv" or "sendonly", choose the media format to send as the most preferred media format from the remote description that is also present in the answer, as described in [RFC3264], Sections 6.1 and 7, and start transmitting RTP media once the @@ -3048,140 +3147,62 @@ using the specified format for resiliency purposes, as discussed in [I-D.ietf-rtcweb-fec], Section 3.2. Note that unlike RTX or FEC media formats, the "red" format is transmitted on the Source RTP Stream, not the Redundancy RTP Stream. * Enable the RTCP feedback mechanisms referenced in the media section for all Source RTP Streams using the specified media formats. Specifically, begin or continue sending the requested feedback types and reacting to received feedback, as specified - in [RFC4585], Section 4.2. When sending RTCP feedback, use the - SSRC of an outgoing Source RTP Stream as the RTCP sender SSRC; - if no outgoing Source RTP Stream exists, choose a random one. + in [RFC4585], Section 4.2. When sending RTCP feedback, follow + the rules and recommendations from + [I-D.ietf-avtcore-rtp-multi-stream], Section 5.4.1 to select + which SSRC to use. * If the directional attribute is of type "recvonly" or "inactive", stop transmitting all RTP media, but continue sending RTCP, as described in [RFC3264], Section 5.1. o If the media section proto value indicates use of SCTP: * If no SCTP association yet exists, prepare to initiate a SCTP association over the associated ICE component and DTLS connection, using the local SCTP port value from the local description, and the remote SCTP port value from the remote description, as described in [I-D.ietf-mmusic-sctp-sdp], Section 10.2. If the answer contains valid bundle groups, discard any ICE components for the m= sections that will be bundled onto the primary ICE components in each bundle, and begin muxing these m= sections accordingly, as described in [I-D.ietf-mmusic-sdp-bundle-negotiation], Section 8.2. -6. Processing RTP/RTCP packets - - Note: The following algorithm does not yet have WG consensus but is - included here as something concrete for the working group to discuss. - - When an RTP packet is received by a transport and passes SRTP - authentication, that packet needs to be routed to the correct - RtpReceiver. For each transport, the following steps MUST be - followed to prepare to route packets: - - Construct a table mapping MID to RtpReceiver for each RtpReceiver - configured to receive from this transport. - - Construct a table mapping incoming SSRC to RtpReceiver for each - RtpReceiver configured to receive from this transport and for each - SSRC that RtpReceiver is configured to receive. Some of the SSRCs - may be present in the m= section corresponding to that RtpReceiver - in the remote description. - - Construct a table mapping outgoing SSRC to RtpSender for each - RtpSender configured to transmit from this transport and for each - SSRC that RtpSender is configured to use when sending. - - Construct a table mapping payload type to RtpReceiver for each - RtpReceiver configured to receive from this transport and for each - payload type that RtpReceiver is configured to receive. The - payload types of a given RtpReceiver are found in the m= section - corresponding to that RtpReceiver in the local description. If - any payload type could map to more than one RtpReceiver, map to - the RtpReceiver whose m= section appears earliest in the local - description. - - As RtpTransceivers (and, thus, RtpReceivers) are added, removed, - stopped, or reconfigured, the tables above must also be updated. - - For each RTP packet received, the following steps MUST be followed to - route the packet: - - If the packet has a MID and that MID is not in the table mapping - MID to RtpReceiver, drop the packet and stop. - - If the packet has a MID and that MID is in the table mapping MID - to RtpReceiver, update the incoming SSRC mapping table to include - an entry that maps the packet's SSRC to the RtpReceiver for that - MID. - - If the packet's SSRC is in the incoming SSRC mapping table, - deliver the packet to the associated RtpReceiver and stop. - - If the packet's payload type is in the payload type table, update - the the incoming SSRC mapping table to include an entry that maps - the packet's SSRC to the RtpReceiver for that payload type. In - addition, deliver the packet to the associated RtpReceiver and - stop. - - Otherwise, drop the packet. - - For each RTCP packet received (including each RTCP packet that is - part of a compound RTCP packet), the following type-specific handling - MUST be performed to route the packet: - - If the packet is of type SR, and the sender SSRC for the packet is - found in the incoming SSRC table, deliver a copy of the packet to - the RtpReceiver associated with that SSRC. In addition, for each - report block in the report whose SSRC is found in the outgoing - SSRC table, deliver a copy of the RTCP packet to the RtpSender - associated with that SSRC. - - If the packet is of type RR, for each report block in the packet - whose SSRC is found in the outgoing SSRC table, deliver a copy of - the RTCP packet to the RtpSender associated with that SSRC. - - If the packet is of type SDES, and the sender SSRC for the packet - is found in the incoming SSRC table, deliver the packet to the - RtpReceiver associated with that SSRC. In addition, for each - chunk in the packet that contains a MID that is in the table - mapping MID to RtpReceiver, update the incoming SSRC mapping table - to include an entry that maps the SSRC for that chunk to the - RtpReceiver associated with that MID. (This case can occur when - RTCP for a source is received before any RTP packets.) + If the description is of type "answer", and there are still remaining + candidates in the ICE candidate pool, discard them. - If the packet is of type BYE, for each SSRC indicated in the - packet that is found in the incoming SSRC table, deliver a copy of - the packet to the RtpReceiver associated with that SSRC. +6. Processing RTP/RTCP - If the packet is of type RTPFB or PSFB, as defined in [RFC4585], - and the media source SSRC for the packet is found in the outgoing - SSRC table, deliver the packet to the RtpSender associated with - that SSRC. + When bundling, associating incoming RTP/RTCP with the proper m= + section is defined in [I-D.ietf-mmusic-sdp-bundle-negotiation]. [The + BUNDLE draft does not currently contain the necessary text to + describe this demux, but when it does it will contain text like that + contained in Appendix B.] When not bundling, the proper m= section + is clear from the ICE component over which the RTP/RTCP is received. - After packets are routed to the RtpReceiver, further processing of - the RTP packets is done at the RtpReceiver level. This includes - using [I-D.ietf-mmusic-rid] to distinguish between multiple Encoded - Streams, as well as determine which Source RTP stream should be - repaired by a given Redundancy RTP stream. If the RTP packet's PT - does not match any codec in use by the RtpReceiver, the packet will - be dropped. + Once the proper m= section(s) are known, RTP/RTCP is delivered to the + RtpTransceiver(s) associated with the m= section(s) and further + processing of the RTP/RTCP is done at the RtpTransceiver level. This + includes using RID [I-D.ietf-mmusic-rid] to distinguish between + multiple Encoded Streams, as well as determine which Source RTP + stream should be repaired by a given Redundancy RTP stream. 7. Examples Note that this example section shows several SDP fragments. To format in 72 columns, some of the lines in SDP have been split into multiple lines, where leading whitespace indicates that a line is a continuation of the previous line. In addition, some blank lines have been added to improve readability but are not valid in SDP. More examples of SDP for WebRTC call flows can be found in @@ -3785,20 +3806,26 @@ Significant text incorporated in the draft as well and review was provided by Peter Thatcher, Taylor Brandstetter, Harald Alvestrand and Suhas Nandakumar. Dan Burnett, Neil Stratford, Anant Narayanan, Andrew Hutton, Richard Ejzak, Adam Bergkvist and Matthew Kaufman all provided valuable feedback on this proposal. 11. References 11.1. Normative References + [I-D.ietf-avtcore-rtp-multi-stream] + Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, + "Sending Multiple RTP Streams in a Single RTP Session", + draft-ietf-avtcore-rtp-multi-stream-11 (work in progress), + December 2015. + [I-D.ietf-avtext-rid] Roach, A., Nandakumar, S., and P. Thatcher, "RTP Stream Identifier (RID) Source Description (SDES)", draft-ietf- avtext-rid-00 (work in progress), February 2016. [I-D.ietf-ice-trickle] Ivov, E., Rescorla, E., Uberti, J., and P. Saint-Andre, "Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol". @@ -3872,25 +3899,20 @@ [I-D.ietf-rtcweb-security-arch] Rescorla, E., "WebRTC Security Architecture", draft-ietf- rtcweb-security-arch-09 (work in progress), February 2014. [I-D.ietf-rtcweb-video] Roach, A., "WebRTC Video Processing and Codec Requirements", draft-ietf-rtcweb-video-00 (work in progress), July 2014. - [I-D.nandakumar-mmusic-proto-iana-registration] - Nandakumar, S., "IANA registration of SDP 'proto' - attribute for transporting RTP Media over TCP under - various RTP profiles.", September 2014. - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June @@ -3943,35 +3965,45 @@ Attributes in the Session Description Protocol (SDP)", RFC 6236, May 2011. [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security Version 1.2", RFC 6347, January 2012. [RFC6904] Lennox, J., "Encryption of Header Extensions in the Secure Real-time Transport Protocol (SRTP)", RFC 6904, April 2013. + [RFC7850] Nandakumar, S., "Registering Values of the SDP 'proto' + Field for Transporting RTP Media over TCP under Various + RTP Profiles", RFC 7850, DOI 10.17487/RFC7850, April 2016, + . + 11.2. Informative References [I-D.ietf-rtcweb-ip-handling] Uberti, J. and G. Shieh, "WebRTC IP Address Handling Recommendations", draft-ietf-rtcweb-ip-handling-01 (work in progress), March 2016. [I-D.nandakumar-rtcweb-sdp] Nandakumar, S. and C. Jennings, "SDP for the WebRTC", draft-nandakumar-rtcweb-sdp-02 (work in progress), July 2013. [RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for Comfort Noise (CN)", RFC 3389, September 2002. + [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. + Jacobson, "RTP: A Transport Protocol for Real-Time + Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, + July 2003, . + [RFC3556] Casner, S., "Session Description Protocol (SDP) Bandwidth Modifiers for RTP Control Protocol (RTCP) Bandwidth", RFC 3556, July 2003. [RFC3960] Camarillo, G. and H. Schulzrinne, "Early Media and Ringing Tone Generation in the Session Initiation Protocol (SIP)", RFC 3960, December 2004. [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session Description Protocol (SDP) Security Descriptions for Media @@ -4002,20 +4034,26 @@ [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer Security (DTLS) Extension to Establish Keys for the Secure Real-time Transport Protocol (SRTP)", RFC 5764, May 2010. [RFC6464] Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time Transport Protocol (RTP) Header Extension for Client-to- Mixer Audio Level Indication", RFC 6464, DOI 10.17487/RFC6464, December 2011, . + [RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and + B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms + for Real-Time Transport Protocol (RTP) Sources", RFC 7656, + DOI 10.17487/RFC7656, November 2015, + . + [W3C.WD-webrtc-20140617] Bergkvist, A., Burnett, D., Narayanan, A., and C. Jennings, "WebRTC 1.0: Real-time Communication Between Browsers", World Wide Web Consortium WD WD-webrtc- 20140617, June 2014, . Appendix A. Appendix A For the syntax validation performed in Section 5.7, the following @@ -4053,24 +4091,209 @@ | option) | | | msid | [I-D.ietf-mmusic-msid] Section 2 | | rid | [I-D.ietf-mmusic-rid] Section 10 | | simulcast | [I-D.ietf-mmusic-sdp-simulcast]Section | | | 6.1 | | dtls-id | [I-D.ietf-mmusic-dtls-sdp]Section 4 | +-----------------------+-------------------------------------------+ Table 1: SDP ABNF References -Appendix B. Change log +Appendix B. Appendix B + + The following text is meant to completely replace section + "Associating RTP/RTCP Streams With Correct SDP Media Description" of + [I-D.ietf-mmusic-sdp-bundle-negotiation]. + + As described in [RFC3550], RTP/RTCP packets are associated with RTP + streams as defined in [RFC7656]. Each RTP stream is identified by an + SSRC value, and each RTP/RTCP packet carries an SSRC value that is + used to associate the packet with the correct RTP stream. An RTCP + packet can carry multiple SSRC values, and might therefore be + associated with multiple RTP streams. + + In order to be able to process received RTP/RTCP packets correctly it + must be possible to associate an RTP stream with the correct "m=" + line, as the "m=" line and SDP attributes associated with the "m=" + line contain information needed to process the packets. + + As all RTP streams associated with a BUNDLE group use the same + address:port combination for sending and receiving RTP/RTCP packets, + the local address:port combination cannot be used to associate an RTP + stream with the correct "m=" line. In addition, multiple RTP streams + might be associated with the same "m=" line. + + An offerer and answerer can inform each other which SSRC values they + will use for an RTP stream by using the SDP 'ssrc' attribute + [RFC5576]. However, an offerer will not know which SSRC values the + answerer will use until the offerer has received the answer providing + that information. Due to this, before the offerer has received the + answer, the offerer will not be able to associate an RTP stream with + the correct "m=" line using the SSRC value associated with the RTP + stream. In addition, the offerer and answerer may start using new + SSRC values mid-session, without informing each other using the SDP + 'ssrc' attribute. + + In order for an offerer and answerer to always be able to associate + an RTP stream with the correct "m=" line, the offerer and answerer + using the BUNDLE extension MUST support the mechanism defined in + [I-D.ietf-mmusic-sdp-bundle-negotiation] section 14. where the + offerer and answerer insert the identification-tag associated with an + "m=" line (provided by the remote peer) into RTP and RTCP packets + associated with a BUNDLE group. + + The mapping from an SSRC to an identification-tag is carried in RTCP + SDES packets or in RTP header extensions + ([I-D.ietf-mmusic-sdp-bundle-negotiation] section 14). Since a + compound RTCP packet can contain multiple RTCP SDES packets, and each + RTCP SDES packet can contain multiple chunks, an RTCP packet can + contain several SSRC to identification-tag mappings. The offerer and + answerer maintain tables used for routing that are updated each time + an RTP/RTCP packet contains new information that affects how packets + should be routed. + + To prepare for demultiplexing RTP packets to the correct "m=" line, + the following steps MUST be followed for each BUNDLE group. + + Construct a table mapping MID to "m=" line for each "m=" line in + this BUNDLE group. Note that an "m=" line may only have one MID. + + Construct a table mapping incoming SSRC to "m=" line for each "m=" + line in this BUNDLE group and for each SSRC configured for + receiving in that "m=" line. + + Construct a table mapping outgoing SSRC to "m=line" for each "m=" + line in this BUNDLE group and for each SSRC configured for sending + in that "m=" line. + + Construct a table mapping payload type to "m=" line for each "m=" + line in the BUNDLE group and for each payload type configured for + receiving in that "m=" line. If any payload type is configured + for receiving in more than one "m=" line in the BUNDLE group, do + not it include it in the table. + + Note that for each of these tables, there can only be one mapping + for any given key (MID, SSRC, or PT). In other words, the tables + are not multimaps. + + As "m=" lines are added or removed from the BUNDLE groups, or their + configurations are changed, the tables above MUST also be updated. + + For each RTP packet received, the following steps MUST be followed to + route the packet to the correct "m=" section within a BUNDLE group. + Note that the phrase 'deliver a packet to the "m=" line' means to + further process the packet as would normally happen with RTP/RTCP, if + it were received on a transport associated with that "m=" line + outside of a BUNDLE group (i.e., if the "m=" line were not BUNDLEd), + including dropping an RTP packet if the packet's PT does not match + any PT in the "m=" line. + + If the packet has a MID and that MID is not in the table mapping + MID to "m=" line, drop the packet and stop. + + If the packet has a MID and that MID is in the table mapping MID + to "m=" line, update the incoming SSRC mapping table to include an + entry that maps the packet's SSRC to the "m=" line for that MID. + + If the packet's SSRC is in the incoming SSRC mapping table, route + the packet to the associated "m=" line and stop. + + If the packet's payload type is in the payload type table, update + the the incoming SSRC mapping table to include an entry that maps + the packet's SSRC to the "m=" line for that payload type. In + addition, route the packet to the associated "m=" line and stop. + + Otherwise, drop the packet. + + For each RTCP packet received (including each RTCP packet that is + part of a compound RTCP packet), the packet MUST be routed to the + appropriate handler for the SSRCs it contains information about. + Some examples of such handling are given below. + + If the packet is of type SR, and the sender SSRC for the packet is + found in the incoming SSRC table, deliver a copy of the packet to + the "m=" line associated with that SSRC. In addition, for each + report block in the report whose SSRC is found in the outgoing + SSRC table, deliver a copy of the RTCP packet to the "m=" line + associated with that SSRC. + + If the packet is of type RR, for each report block in the packet + whose SSRC is found in the outgoing SSRC table, deliver a copy of + the RTCP packet to the "m=" line associated with that SSRC. + + If the packet is of type SDES, and the sender SSRC for the packet + is found in the incoming SSRC table, deliver the packet to the + "m=" line associated with that SSRC. In addition, for each chunk + in the packet that contains a MID that is in the table mapping MID + to "m=" line, update the incoming SSRC mapping table to include an + entry that maps the SSRC for that chunk to the "m=" line + associated with that MID. (This case can occur when RTCP for a + source is received before any RTP packets.) + + If the packet is of type BYE, for each SSRC indicated in the + packet that is found in the incoming SSRC table, deliver a copy of + the packet to the "m=" line associated with that SSRC. + + If the packet is of type RTPFB or PSFB, as defined in [RFC4585], + and the media source SSRC for the packet is found in the outgoing + SSRC table, deliver the packet to the "m=" line associated with + that SSRC. + +Appendix C. Change log Note: This section will be removed by RFC Editor before publication. + Changes in draft-18: + + o Update demux algorithm and move it to an appendix in preparation + for merging it into BUNDLE. + + o Clarify why we can't handle an incoming offer to send simulcast. + + o Expand IceCandidate object text. + + o Further document use of ICE candidate pool. + + o Document removeTrack. + + o Update requirements to only accept the last generated offer/answer + as an argument to setLocalDescription. + + o Allow round pixels. + + o Fix code around default timing when AVPF is not specified. + + o Clean up terminology around m= line and m=section. + + o Provide a more realistic example for minimum decoder capabilities. + + o Document behavior when rtcp-mux policy is require but rtcp-mux + attribute not provided. + + o Expanded discussion of RtpSender and RtpReceiver. + + o Add RtpTransceiver.currentDirection and document setDirection. + + o Require imageattr x=0, y=0 to indicate that there are no valid + resolutions. + + o Require a privacy-preserving MID/RID construction. + + o Require support for RFC 3556 bandwidth modifiers. + + o Update maxptime description. + + o Note that endpoints may encounter extra codecs in answers and + subsequent offers from non-JSEP peers. + + o Update references. + Changes in draft-17: o Split createOffer and createAnswer sections to clearly indicate attributes which always appear and which only appear when not bundled into another m= section. o Add descriptions of RtpTransceiver methods. o Describe how to process RTCP feedback attributes. @@ -4172,33 +4395,33 @@ o Remove unused references. o Remove text advocating use of unilateral PTs. o Trigger an ICE restart even if the ICE candidate policy is being made more strict. o Remove the 'public' ICE candidate policy. - o Move open issues/TODOs into GitHub issues. + o Move open issues into GitHub issues. o Split local/remote description accessors into current/pending. o Clarify a=imageattr handling. o Add more detail on VoiceActivityDetection handling. o Reference draft-shieh-rtcweb-ip-handling. o Make it clear when an ICE restart should occur. - o Resolve reference TODOs. + o Resolve changes needed in references. o Remove MSID semantics. o ice-options are now at session level. o Default RTCP mux policy is now 'require'. Changes in draft-12: o Filled in sections on applying local and remote descriptions. @@ -4219,21 +4442,25 @@ Changes in draft-11: o Clarified handling of RTP CNAMEs. o Updated what SDP lines should be processed or ignored. o Specified how a=imageattr should be used. Changes in draft-10: - o TODO + o Described video size negotiation with imageattr. + + o Clarified rejection of sections that do not have mux-only. + + o Add handling of LS groups Changes in draft-09: o Don't return null for {local,remote}Description after close(). o Changed TCP/TLS to UDP/DTLS in RTP profile names. o Separate out bundle and mux policy. o Added specific references to FEC mechanisms.