--- 1/draft-ietf-rtcweb-video-02.txt 2014-11-25 16:14:54.084567311 -0800 +++ 2/draft-ietf-rtcweb-video-03.txt 2014-11-25 16:14:54.104567800 -0800 @@ -1,18 +1,18 @@ Network Working Group A.B. Roach Internet-Draft Mozilla -Intended status: Standards Track October 27, 2014 -Expires: April 30, 2015 +Intended status: Standards Track November 25, 2014 +Expires: May 29, 2015 WebRTC Video Processing and Codec Requirements - draft-ietf-rtcweb-video-02 + draft-ietf-rtcweb-video-03 Abstract This specification provides the requirements and considerations for WebRTC applications to send and receive video across a network. It specifies the video processing that is required, as well as video codecs and their parameters. Status of This Memo @@ -22,21 +22,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on April 30, 2015. + This Internet-Draft will expire on May 29, 2015. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -46,33 +46,32 @@ the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2 3. Pre and Post Processing . . . . . . . . . . . . . . . . . . . 2 3.1. Camera Source Video . . . . . . . . . . . . . . . . . . . 3 3.2. Screen Source Video . . . . . . . . . . . . . . . . . . . 3 - 4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 4 - 5. Codec-Specific Considerations . . . . . . . . . . . . . . . . 4 - 5.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 5.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 6. Mandatory to Implement Video Codec . . . . . . . . . . . . . 6 - 6.1. Temperature of Working Group . . . . . . . . . . . . . . 6 - 7. Security Considerations . . . . . . . . . . . . . . . . . . . 7 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 - 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7 + 4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 3 + 5. Mandatory to Implement Video Codec . . . . . . . . . . . . . 4 + 6. Codec-Specific Considerations . . . . . . . . . . . . . . . . 5 + 6.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 6.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 7. Security Considerations . . . . . . . . . . . . . . . . . . . 6 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 + 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 6 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 10.1. Normative References . . . . . . . . . . . . . . . . . . 7 - 10.2. Informative References . . . . . . . . . . . . . . . . . 9 - Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9 + 10.2. Informative References . . . . . . . . . . . . . . . . . 8 + Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 8 1. Introduction One of the major functions of WebRTC endpoints is the ability to send and receive interactive video. The video might come from a camera, a screen recording, a stored file, or some other source. This specification defines how the video is used and discusses special considerations for processing the video. It also covers the video- related algorithms WebRTC devices need to support. @@ -125,104 +124,125 @@ be prepared to handle mid-stream resolution changes in a way that preserves their utility. Precise handling (e.g., resizing the element a video is rendered in versus scaling down the received stream; decisions around letter/pillarboxing) is left to the discretion of the application. Additionally, attention is drawn to the requirements in [I-D.ietf-rtcweb-security-arch] section 5.2 and the considerations in [I-D.ietf-rtcweb-security] section 4.1.1. - TODO: Do we want to define additional metadata to indicate whether a - stream is sourced from a camera versus a screen capture? This would - allow the receiving party to tune, e.g., output filters. It would - appear that H.263 has this kind of indicator built into its - bitstream, but I found no analog in H.264 or VP8. - 4. Stream Orientation In some circumstances - and notably those involving mobile devices - the orientation of the camera may not match the orientation used by the encoder. Of more importance, the orientation may change over the course of a call, requiring the receiver to change the orientation in which it renders the stream. While the sender may elect to simply change the pre-encoding orientation of frames, this may not be practical or efficient (in particular, in cases where the interface to the camera returns pre- compressed video frames). Note that the potential for this behavior adds another set of circumstances under which the resolution of a screen might change in the middle of a video stream, in addition to those mentioned under "Screen Sourced Video," above. - To accommodate these circumstances, RTCWEB implementations SHOULD - support generating and receiving the R0 and R1 bits of the - Coordination of Video Orientation (CVO) mechanism described in - section 7.4.5 of [TS26.114]. (TODO: Is "SHOULD support" the right - level here?) They MAY support the other bits in the CVO extension, - including the higher-resolution rotation bits. + To accommodate these circumstances, RTCWEB implementations that can + generate media in orientations other than the default MUST support + generating the R0 and R1 bits of the Coordination of Video + Orientation (CVO) mechanism described in section 7.4.5 of [TS26.114], + and MUST send them for all orientations when the peer indicates + support for the mechanism. They MAY support sending the other bits + in the CVO extension, including the higher-resolution rotation bits. + All implementations SHOULD support interpretation of the R0 and R1 + bits, and MAY support the other CVO bits. Further, some codecs support in-band signaling of orientation (for example, the SEI "Display Orientation" messages in H.264 and H.265). If CVO has been negotiated, then the sender MUST NOT make use of such codec-specific mechanisms. However, when support for CVO is not signaled in the SDP, then such implementations MAY make use of the codec-specific mechanisms instead. -5. Codec-Specific Considerations +5. Mandatory to Implement Video Codec - WebRTC endpoints are not required to support the codecs mentioned in - this section. + For the definitions of "WebRTC Brower," "WebRTC Non-Browser", and + "WebRTC-Compatible Endpoint" as they are used in this section, please + refer to [I-D.ietf-rtcweb-overview]. - However, to foster interoperability between endpoints that have - codecs in common, if they do support one of the listed codecs, then - they need to meet the requirements specified in the subsection for - that codec. + WebRTC Browsers MUST implement the VP8 video codec as described in + [RFC6386] and H.264 as described in [H264]. + + WebRTC Non-Browsers that support transmitting and/or receiving video + MUST implement the VP8 video codec as described in [RFC6386] and + H.264 as described in [H264]. + + To promote the use of non-royalty bearing video codecs, + participants in the RTCWEB working group, and any successor + working groups in the IETF, intend to monitor the evolving + licensing landscape as it pertains to the two mandatory-to- + implement codecs. If compelling evidence arises that one of the + codecs is available for use on a royalty-free basis, the working + group plans to revisit the question of which codecs are required + for Non-Browsers, with the intention being that the royalty-free + codec will remain mandatory to implement, and the other will + become optional. + + These provisions apply to WebRTC Non-Browsers only. There is no + plan to revisit the codecs required for WebRTC Browsers. + + "WebRTC-compatible endpoints" are free to implement any video codecs + they see fit. This follows logically from the definition of "WebRTC- + compatible endpoint." It is, of course, advisable to implement at + least one of the video codecs that is mandated for WebRTC Browsers, + and implementors are encouraged to do so. + +6. Codec-Specific Considerations SDP allows for codec-independent indication of preferred video resolutions using the mechanism described in [RFC6236]. If a recipient of video indicates a receiving resolution, the sender SHOULD accommodate this resolution, as the receiver may not be capable of handling higher resolutions. Additionally, codecs may include codec-specific means of signaling maximum receiver abilities with regards to resolution, frame rate, and bitrate. Unless otherwise signaled in SDP, recipients of video streams are MUST be able to decode video at a rate of at least 20 fps at a resolution of at least 320x240. These values are selected based on the recommendations in [HSUP1]. Encoders are encouraged to support encoding media with at least the same resolution and frame rates cited above. -5.1. VP8 +6.1. VP8 - If VP8, defined in [RFC6386], is supported, then the endpoint MUST - support the payload formats defined in [I-D.ietf-payload-vp8]. In - addition it MUST support the 'bilinear' and 'none' reconstruction - filters. + For the VP8 codec, defined in [RFC6386], endpoints MUST support the + payload formats defined in [I-D.ietf-payload-vp8]. In addition they + MUST support the 'bilinear' and 'none' reconstruction filters. TODO: There have been claims that VP8 already requires supporting both filters; if true, these do not need to be reiterated here. In addition to the [RFC6236] mechanism, VP8 encoders MUST limit the streams they send to conform to the values indicated by receivers in the corresponding max-fr and max-fs SDP attributes. -5.2. H.264 +6.2. H.264 - If [H264] is supported, then the device MUST support the payload - formats defined in [RFC6184]. In addition, they MUST support - Constrained Baseline Profile Level 1.2, and they SHOULD support H.264 - Constrained High Profile Level 1.3. + For the [H264] codec, endpoints MUST support the payload formats + defined in [RFC6184]. In addition, they MUST support Constrained + Baseline Profile Level 1.2, and they SHOULD support H.264 Constrained + High Profile Level 1.3. Implementations of the H.264 codec have utilized a wide variety of optional parameters. To improve interoperability the following parameter settings are specified: packetization-mode: Packetization-mode 1 MUST be supported. Other modes MAY be negotiated and used. profile-level-id: Implementations MUST include this parameter within SDP and SHOULD interpret it when receiving it. @@ -236,71 +256,20 @@ the highest quality of video possible. sprop-parameter-sets: H.264 allows sequence and picture information to be sent both in-band, and out-of-band. WebRTC implementations MUST signal this information in-band; as a result, this parameter will not be present in SDP. TODO: Do we need to require the handling of specific SEI messages? One example that has been raised is freeze-frame messages. -6. Mandatory to Implement Video Codec - - Note: This section is here purely as a placeholder, as there is not - yet WG Consensus on Mandatory to Implement video codecs. The issue - is more complicated than may be immediately apparent to newcomers, - who are strongly encouraged to familiarize themselves with the - previous discussions on the topic before engaging on this issue. - - The currently recorded working group consensus is that all - implementations MUST support a single, specified mandatory-to- - implement codec. The remaining decision point is a selection of this - single codec. - -6.1. Temperature of Working Group - - To capture the conversation so far, this section summarizes the - result of a straw poll that the working group undertook in December - 2013 and January 2014. Respondents were asked to answer "Yes," - "Acceptable," or "No" for each option. The options were collected - from the working group at large prior to the initiation of the straw - poll. - - Yes Acc No - --- --- --- - 1. All entities MUST support H.264 48% 11% 41% - 2. All entities MUST support VP8 41% 17% 42% - 3. All entities MUST support both H.264 and VP8 9% 38% 53% - 4. Browsers MUST support both H.264 and VP8, other - entities MUST support at least one of H.264 - and VP8 11% 34% 55% - 5. All entities MUST support at least one of - H.264 and VP8 10% 16% 74% - 6. All entities MUST support H.261 5% 23% 72% - 7. There is no MTI video codec 12% 30% 58% - 8. All entities MUST support H.261 and all entities - MUST support at least one of H.264 and VP8 4% 28% 68% - 9. All entities MUST support Theora 7% 26% 67% - - 10. All entities MUST implement at least two of - {VP8, H.264, H.261} 5% 30% 65% - 11. All entities MUST implement at least two of - {VP8, H.264, H.263} 5% 25% 70% - 12. All entities MUST support decoding using both - H.264 and VP8, and MUST support encoding using - at least one of H.264 or VP8 7% 20% 73% - 13. All entities MUST support H.263 6% 19% 75% - 14. All entities MUST implement at least two of - {VP8, H.264, Theora} 6% 27% 67% - 15. All entities MUST support decoding using Theora 1% 15% 84% - 16. All entities MUST support Motion JPEG 1% 25% 74% - 7. Security Considerations This specification does not introduce any new mechanisms or security concerns beyond what the other documents it references. In WebRTC, video is protected using DTLS/SRTP. A complete discussion of the security can be found in [I-D.ietf-rtcweb-security] and [I-D.ietf-rtcweb-security-arch]. Implementers should consider whether the use of variable bit rate video codecs are appropriate for their application based on [RFC6562]. @@ -324,20 +293,25 @@ [HSUP1] ITU-T Recommendation H.Sup1, "Application profile - Sign language and lip-reading real-time conversation using low bit rate video communication", May 1999. [I-D.ietf-payload-vp8] Westin, P., Lundin, H., Glover, M., Uberti, J., and F. Galligan, "RTP Payload Format for VP8 Video", draft-ietf- payload-vp8-11 (work in progress), February 2014. + [I-D.ietf-rtcweb-overview] + Alvestrand, H., "Overview: Real Time Protocols for + Browser-based Applications", draft-ietf-rtcweb-overview-12 + (work in progress), October 2014. + [IEC23001-8] ISO/IEC 23001-8:2013/DCOR1, "Coding independent media description code points", 2013. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for Uncompressed Video", RFC 4175, September 2005.