draft-ietf-rtcweb-video-02.txt		draft-ietf-rtcweb-video-03.txt
	Network Working Group A.B. Roach		Network Working Group A.B. Roach
	Internet-Draft Mozilla		Internet-Draft Mozilla
	Intended status: Standards Track October 27, 2014		Intended status: Standards Track November 25, 2014
	Expires: April 30, 2015		Expires: May 29, 2015
	WebRTC Video Processing and Codec Requirements		WebRTC Video Processing and Codec Requirements
	draft-ietf-rtcweb-video-02		draft-ietf-rtcweb-video-03
	Abstract		Abstract
	This specification provides the requirements and considerations for		This specification provides the requirements and considerations for
	WebRTC applications to send and receive video across a network. It		WebRTC applications to send and receive video across a network. It
	specifies the video processing that is required, as well as video		specifies the video processing that is required, as well as video
	codecs and their parameters.		codecs and their parameters.
	Status of This Memo		Status of This Memo
	skipping to change at page 1, line 33		skipping to change at page 1, line 33
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF). Note that other groups may also distribute		Task Force (IETF). Note that other groups may also distribute
	working documents as Internet-Drafts. The list of current Internet-		working documents as Internet-Drafts. The list of current Internet-
	Drafts is at http://datatracker.ietf.org/drafts/current/.		Drafts is at http://datatracker.ietf.org/drafts/current/.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	This Internet-Draft will expire on April 30, 2015.		This Internet-Draft will expire on May 29, 2015.
	Copyright Notice		Copyright Notice
	Copyright (c) 2014 IETF Trust and the persons identified as the		Copyright (c) 2014 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(http://trustee.ietf.org/license-info) in effect on the date of		(http://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	skipping to change at page 2, line 12		skipping to change at page 2, line 12
	the Trust Legal Provisions and are provided without warranty as		the Trust Legal Provisions and are provided without warranty as
	described in the Simplified BSD License.		described in the Simplified BSD License.
	Table of Contents		Table of Contents
	1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2		1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
	2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2		2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 2
	3. Pre and Post Processing . . . . . . . . . . . . . . . . . . . 2		3. Pre and Post Processing . . . . . . . . . . . . . . . . . . . 2
	3.1. Camera Source Video . . . . . . . . . . . . . . . . . . . 3		3.1. Camera Source Video . . . . . . . . . . . . . . . . . . . 3
	3.2. Screen Source Video . . . . . . . . . . . . . . . . . . . 3		3.2. Screen Source Video . . . . . . . . . . . . . . . . . . . 3
	4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 4		4. Stream Orientation . . . . . . . . . . . . . . . . . . . . . 3
	5. Codec-Specific Considerations . . . . . . . . . . . . . . . . 4		5. Mandatory to Implement Video Codec . . . . . . . . . . . . . 4
	5.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5		6. Codec-Specific Considerations . . . . . . . . . . . . . . . . 5
	5.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 5		6.1. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
	6. Mandatory to Implement Video Codec . . . . . . . . . . . . . 6		6.2. H.264 . . . . . . . . . . . . . . . . . . . . . . . . . . 5
	6.1. Temperature of Working Group . . . . . . . . . . . . . . 6		7. Security Considerations . . . . . . . . . . . . . . . . . . . 6
	7. Security Considerations . . . . . . . . . . . . . . . . . . . 7		8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6
	8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7		9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 6
	9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7
	10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7		10. References . . . . . . . . . . . . . . . . . . . . . . . . . 7
	10.1. Normative References . . . . . . . . . . . . . . . . . . 7		10.1. Normative References . . . . . . . . . . . . . . . . . . 7
	10.2. Informative References . . . . . . . . . . . . . . . . . 9		10.2. Informative References . . . . . . . . . . . . . . . . . 8
	Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 9		Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 8
	1. Introduction		1. Introduction
	One of the major functions of WebRTC endpoints is the ability to send		One of the major functions of WebRTC endpoints is the ability to send
	and receive interactive video. The video might come from a camera, a		and receive interactive video. The video might come from a camera, a
	screen recording, a stored file, or some other source. This		screen recording, a stored file, or some other source. This
	specification defines how the video is used and discusses special		specification defines how the video is used and discusses special
	considerations for processing the video. It also covers the video-		considerations for processing the video. It also covers the video-
	related algorithms WebRTC devices need to support.		related algorithms WebRTC devices need to support.
	skipping to change at page 3, line 42		skipping to change at page 3, line 42
	be prepared to handle mid-stream resolution changes in a way that		be prepared to handle mid-stream resolution changes in a way that
	preserves their utility. Precise handling (e.g., resizing the		preserves their utility. Precise handling (e.g., resizing the
	element a video is rendered in versus scaling down the received		element a video is rendered in versus scaling down the received
	stream; decisions around letter/pillarboxing) is left to the		stream; decisions around letter/pillarboxing) is left to the
	discretion of the application.		discretion of the application.
	Additionally, attention is drawn to the requirements in		Additionally, attention is drawn to the requirements in
	[I-D.ietf-rtcweb-security-arch] section 5.2 and the considerations in		[I-D.ietf-rtcweb-security-arch] section 5.2 and the considerations in
	[I-D.ietf-rtcweb-security] section 4.1.1.		[I-D.ietf-rtcweb-security] section 4.1.1.
	TODO: Do we want to define additional metadata to indicate whether a
	stream is sourced from a camera versus a screen capture? This would
	allow the receiving party to tune, e.g., output filters. It would
	appear that H.263 has this kind of indicator built into its
	bitstream, but I found no analog in H.264 or VP8.
	4. Stream Orientation		4. Stream Orientation
	In some circumstances - and notably those involving mobile devices -		In some circumstances - and notably those involving mobile devices -
	the orientation of the camera may not match the orientation used by		the orientation of the camera may not match the orientation used by
	the encoder. Of more importance, the orientation may change over the		the encoder. Of more importance, the orientation may change over the
	course of a call, requiring the receiver to change the orientation in		course of a call, requiring the receiver to change the orientation in
	which it renders the stream.		which it renders the stream.
	While the sender may elect to simply change the pre-encoding		While the sender may elect to simply change the pre-encoding
	orientation of frames, this may not be practical or efficient (in		orientation of frames, this may not be practical or efficient (in
	particular, in cases where the interface to the camera returns pre-		particular, in cases where the interface to the camera returns pre-
	compressed video frames). Note that the potential for this behavior		compressed video frames). Note that the potential for this behavior
	adds another set of circumstances under which the resolution of a		adds another set of circumstances under which the resolution of a
	screen might change in the middle of a video stream, in addition to		screen might change in the middle of a video stream, in addition to
	those mentioned under "Screen Sourced Video," above.		those mentioned under "Screen Sourced Video," above.
	To accommodate these circumstances, RTCWEB implementations SHOULD		To accommodate these circumstances, RTCWEB implementations that can
	support generating and receiving the R0 and R1 bits of the		generate media in orientations other than the default MUST support
	Coordination of Video Orientation (CVO) mechanism described in		generating the R0 and R1 bits of the Coordination of Video
	section 7.4.5 of [TS26.114]. (TODO: Is "SHOULD support" the right		Orientation (CVO) mechanism described in section 7.4.5 of [TS26.114],
	level here?) They MAY support the other bits in the CVO extension,		and MUST send them for all orientations when the peer indicates
	including the higher-resolution rotation bits.		support for the mechanism. They MAY support sending the other bits
			in the CVO extension, including the higher-resolution rotation bits.
			All implementations SHOULD support interpretation of the R0 and R1
			bits, and MAY support the other CVO bits.
	Further, some codecs support in-band signaling of orientation (for		Further, some codecs support in-band signaling of orientation (for
	example, the SEI "Display Orientation" messages in H.264 and H.265).		example, the SEI "Display Orientation" messages in H.264 and H.265).
	If CVO has been negotiated, then the sender MUST NOT make use of such		If CVO has been negotiated, then the sender MUST NOT make use of such
	codec-specific mechanisms. However, when support for CVO is not		codec-specific mechanisms. However, when support for CVO is not
	signaled in the SDP, then such implementations MAY make use of the		signaled in the SDP, then such implementations MAY make use of the
	codec-specific mechanisms instead.		codec-specific mechanisms instead.
	5. Codec-Specific Considerations		5. Mandatory to Implement Video Codec
	WebRTC endpoints are not required to support the codecs mentioned in		For the definitions of "WebRTC Brower," "WebRTC Non-Browser", and
	this section.		"WebRTC-Compatible Endpoint" as they are used in this section, please
			refer to [I-D.ietf-rtcweb-overview].
	However, to foster interoperability between endpoints that have		WebRTC Browsers MUST implement the VP8 video codec as described in
	codecs in common, if they do support one of the listed codecs, then		[RFC6386] and H.264 as described in [H264].
	they need to meet the requirements specified in the subsection for
	that codec.		WebRTC Non-Browsers that support transmitting and/or receiving video
			MUST implement the VP8 video codec as described in [RFC6386] and
			H.264 as described in [H264].
			To promote the use of non-royalty bearing video codecs,
			participants in the RTCWEB working group, and any successor
			working groups in the IETF, intend to monitor the evolving
			licensing landscape as it pertains to the two mandatory-to-
			implement codecs. If compelling evidence arises that one of the
			codecs is available for use on a royalty-free basis, the working
			group plans to revisit the question of which codecs are required
			for Non-Browsers, with the intention being that the royalty-free
			codec will remain mandatory to implement, and the other will
			become optional.
			These provisions apply to WebRTC Non-Browsers only. There is no
			plan to revisit the codecs required for WebRTC Browsers.
			"WebRTC-compatible endpoints" are free to implement any video codecs
			they see fit. This follows logically from the definition of "WebRTC-
			compatible endpoint." It is, of course, advisable to implement at
			least one of the video codecs that is mandated for WebRTC Browsers,
			and implementors are encouraged to do so.
			6. Codec-Specific Considerations
	SDP allows for codec-independent indication of preferred video		SDP allows for codec-independent indication of preferred video
	resolutions using the mechanism described in [RFC6236]. If a		resolutions using the mechanism described in [RFC6236]. If a
	recipient of video indicates a receiving resolution, the sender		recipient of video indicates a receiving resolution, the sender
	SHOULD accommodate this resolution, as the receiver may not be		SHOULD accommodate this resolution, as the receiver may not be
	capable of handling higher resolutions.		capable of handling higher resolutions.
	Additionally, codecs may include codec-specific means of signaling		Additionally, codecs may include codec-specific means of signaling
	maximum receiver abilities with regards to resolution, frame rate,		maximum receiver abilities with regards to resolution, frame rate,
	and bitrate.		and bitrate.
	Unless otherwise signaled in SDP, recipients of video streams are		Unless otherwise signaled in SDP, recipients of video streams are
	MUST be able to decode video at a rate of at least 20 fps at a		MUST be able to decode video at a rate of at least 20 fps at a
	resolution of at least 320x240. These values are selected based on		resolution of at least 320x240. These values are selected based on
	the recommendations in [HSUP1].		the recommendations in [HSUP1].
	Encoders are encouraged to support encoding media with at least the		Encoders are encouraged to support encoding media with at least the
	same resolution and frame rates cited above.		same resolution and frame rates cited above.
	5.1. VP8		6.1. VP8
	If VP8, defined in [RFC6386], is supported, then the endpoint MUST		For the VP8 codec, defined in [RFC6386], endpoints MUST support the
	support the payload formats defined in [I-D.ietf-payload-vp8]. In		payload formats defined in [I-D.ietf-payload-vp8]. In addition they
	addition it MUST support the 'bilinear' and 'none' reconstruction		MUST support the 'bilinear' and 'none' reconstruction filters.
	filters.
	TODO: There have been claims that VP8 already requires supporting		TODO: There have been claims that VP8 already requires supporting
	both filters; if true, these do not need to be reiterated here.		both filters; if true, these do not need to be reiterated here.
	In addition to the [RFC6236] mechanism, VP8 encoders MUST limit the		In addition to the [RFC6236] mechanism, VP8 encoders MUST limit the
	streams they send to conform to the values indicated by receivers in		streams they send to conform to the values indicated by receivers in
	the corresponding max-fr and max-fs SDP attributes.		the corresponding max-fr and max-fs SDP attributes.
	5.2. H.264		6.2. H.264
	If [H264] is supported, then the device MUST support the payload		For the [H264] codec, endpoints MUST support the payload formats
	formats defined in [RFC6184]. In addition, they MUST support		defined in [RFC6184]. In addition, they MUST support Constrained
	Constrained Baseline Profile Level 1.2, and they SHOULD support H.264		Baseline Profile Level 1.2, and they SHOULD support H.264 Constrained
	Constrained High Profile Level 1.3.		High Profile Level 1.3.
	Implementations of the H.264 codec have utilized a wide variety of		Implementations of the H.264 codec have utilized a wide variety of
	optional parameters. To improve interoperability the following		optional parameters. To improve interoperability the following
	parameter settings are specified:		parameter settings are specified:
	packetization-mode: Packetization-mode 1 MUST be supported. Other		packetization-mode: Packetization-mode 1 MUST be supported. Other
	modes MAY be negotiated and used.		modes MAY be negotiated and used.
	profile-level-id: Implementations MUST include this parameter within		profile-level-id: Implementations MUST include this parameter within
	SDP and SHOULD interpret it when receiving it.		SDP and SHOULD interpret it when receiving it.
	skipping to change at page 6, line 15		skipping to change at page 6, line 31
	the highest quality of video possible.		the highest quality of video possible.
	sprop-parameter-sets: H.264 allows sequence and picture information		sprop-parameter-sets: H.264 allows sequence and picture information
	to be sent both in-band, and out-of-band. WebRTC implementations		to be sent both in-band, and out-of-band. WebRTC implementations
	MUST signal this information in-band; as a result, this parameter		MUST signal this information in-band; as a result, this parameter
	will not be present in SDP.		will not be present in SDP.
	TODO: Do we need to require the handling of specific SEI messages?		TODO: Do we need to require the handling of specific SEI messages?
	One example that has been raised is freeze-frame messages.		One example that has been raised is freeze-frame messages.
	6. Mandatory to Implement Video Codec
	Note: This section is here purely as a placeholder, as there is not
	yet WG Consensus on Mandatory to Implement video codecs. The issue
	is more complicated than may be immediately apparent to newcomers,
	who are strongly encouraged to familiarize themselves with the
	previous discussions on the topic before engaging on this issue.
	The currently recorded working group consensus is that all
	implementations MUST support a single, specified mandatory-to-
	implement codec. The remaining decision point is a selection of this
	single codec.
	6.1. Temperature of Working Group
	To capture the conversation so far, this section summarizes the
	result of a straw poll that the working group undertook in December
	2013 and January 2014. Respondents were asked to answer "Yes,"
	"Acceptable," or "No" for each option. The options were collected
	from the working group at large prior to the initiation of the straw
	poll.
	Yes Acc No
	--- --- ---
	1. All entities MUST support H.264 48% 11% 41%
	2. All entities MUST support VP8 41% 17% 42%
	3. All entities MUST support both H.264 and VP8 9% 38% 53%
	4. Browsers MUST support both H.264 and VP8, other
	entities MUST support at least one of H.264
	and VP8 11% 34% 55%
	5. All entities MUST support at least one of
	H.264 and VP8 10% 16% 74%
	6. All entities MUST support H.261 5% 23% 72%
	7. There is no MTI video codec 12% 30% 58%
	8. All entities MUST support H.261 and all entities
	MUST support at least one of H.264 and VP8 4% 28% 68%
	9. All entities MUST support Theora 7% 26% 67%
	10. All entities MUST implement at least two of
	{VP8, H.264, H.261} 5% 30% 65%
	11. All entities MUST implement at least two of
	{VP8, H.264, H.263} 5% 25% 70%
	12. All entities MUST support decoding using both
	H.264 and VP8, and MUST support encoding using
	at least one of H.264 or VP8 7% 20% 73%
	13. All entities MUST support H.263 6% 19% 75%
	14. All entities MUST implement at least two of
	{VP8, H.264, Theora} 6% 27% 67%
	15. All entities MUST support decoding using Theora 1% 15% 84%
	16. All entities MUST support Motion JPEG 1% 25% 74%
	7. Security Considerations		7. Security Considerations
	This specification does not introduce any new mechanisms or security		This specification does not introduce any new mechanisms or security
	concerns beyond what the other documents it references. In WebRTC,		concerns beyond what the other documents it references. In WebRTC,
	video is protected using DTLS/SRTP. A complete discussion of the		video is protected using DTLS/SRTP. A complete discussion of the
	security can be found in [I-D.ietf-rtcweb-security] and		security can be found in [I-D.ietf-rtcweb-security] and
	[I-D.ietf-rtcweb-security-arch]. Implementers should consider		[I-D.ietf-rtcweb-security-arch]. Implementers should consider
	whether the use of variable bit rate video codecs are appropriate for		whether the use of variable bit rate video codecs are appropriate for
	their application based on [RFC6562].		their application based on [RFC6562].
	skipping to change at page 8, line 8		skipping to change at page 7, line 21
	[HSUP1] ITU-T Recommendation H.Sup1, "Application profile - Sign		[HSUP1] ITU-T Recommendation H.Sup1, "Application profile - Sign
	language and lip-reading real-time conversation using low		language and lip-reading real-time conversation using low
	bit rate video communication", May 1999.		bit rate video communication", May 1999.
	[I-D.ietf-payload-vp8]		[I-D.ietf-payload-vp8]
	Westin, P., Lundin, H., Glover, M., Uberti, J., and F.		Westin, P., Lundin, H., Glover, M., Uberti, J., and F.
	Galligan, "RTP Payload Format for VP8 Video", draft-ietf-		Galligan, "RTP Payload Format for VP8 Video", draft-ietf-
	payload-vp8-11 (work in progress), February 2014.		payload-vp8-11 (work in progress), February 2014.
			[I-D.ietf-rtcweb-overview]
			Alvestrand, H., "Overview: Real Time Protocols for
			Browser-based Applications", draft-ietf-rtcweb-overview-12
			(work in progress), October 2014.
	[IEC23001-8]		[IEC23001-8]
	ISO/IEC 23001-8:2013/DCOR1, "Coding independent media		ISO/IEC 23001-8:2013/DCOR1, "Coding independent media
	description code points", 2013.		description code points", 2013.
	[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate		[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
	Requirement Levels", BCP 14, RFC 2119, March 1997.		Requirement Levels", BCP 14, RFC 2119, March 1997.
	[RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for		[RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for
	Uncompressed Video", RFC 4175, September 2005.		Uncompressed Video", RFC 4175, September 2005.
End of changes. 16 change blocks.
	95 lines changed or deleted		69 lines changed or added
This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/