draft-ietf-rmcat-eval-test-00.txt | draft-ietf-rmcat-eval-test-01.txt | |||
---|---|---|---|---|
Network Working Group Z. Sarker | Network Working Group Z. Sarker | |||
Internet-Draft Ericsson AB | Internet-Draft Ericsson AB | |||
Intended status: Informational V. Singh | Intended status: Informational V. Singh | |||
Expires: February 15, 2015 Aalto University | Expires: September 11, 2015 Aalto University | |||
X. Zhu | X. Zhu | |||
M. Ramalho | M. Ramalho | |||
Cisco Systems | Cisco Systems | |||
August 14, 2014 | March 10, 2015 | |||
Test Cases for Evaluating RMCAT Proposals | Test Cases for Evaluating RMCAT Proposals | |||
draft-ietf-rmcat-eval-test-00 | draft-ietf-rmcat-eval-test-01 | |||
Abstract | Abstract | |||
The Real-time Transport Protocol (RTP) is used to transmit media in | The Real-time Transport Protocol (RTP) is used to transmit media in | |||
multimedia telephony applications, these applications are typically | multimedia telephony applications, these applications are typically | |||
required to implement congestion control. The RMCAT working group is | required to implement congestion control. The RMCAT working group is | |||
currently working on candidate algorithms for such interactive real- | currently working on candidate algorithms for such interactive real- | |||
time multimedia applications. This document describes the test cases | time multimedia applications. This document describes the test cases | |||
to be used in the performance evaluation of those candidate | to be used in the performance evaluation of those candidate | |||
algorithms. | algorithms. | |||
skipping to change at page 1, line 40 | skipping to change at page 1, line 40 | |||
Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
Drafts is at http://datatracker.ietf.org/drafts/current/. | Drafts is at http://datatracker.ietf.org/drafts/current/. | |||
Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
This Internet-Draft will expire on February 15, 2015. | This Internet-Draft will expire on September 11, 2015. | |||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2014 IETF Trust and the persons identified as the | Copyright (c) 2015 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
(http://trustee.ietf.org/license-info) in effect on the date of | (http://trustee.ietf.org/license-info) in effect on the date of | |||
publication of this document. Please review these documents | publication of this document. Please review these documents | |||
carefully, as they describe your rights and restrictions with respect | carefully, as they describe your rights and restrictions with respect | |||
to this document. Code Components extracted from this document must | to this document. Code Components extracted from this document must | |||
include Simplified BSD License text as described in Section 4.e of | include Simplified BSD License text as described in Section 4.e of | |||
the Trust Legal Provisions and are provided without warranty as | the Trust Legal Provisions and are provided without warranty as | |||
described in the Simplified BSD License. | described in the Simplified BSD License. | |||
Table of Contents | Table of Contents | |||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 | 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 | |||
3. Structure of Test cases . . . . . . . . . . . . . . . . . . . 3 | 3. Structure of Test cases . . . . . . . . . . . . . . . . . . . 3 | |||
4. Recommended Evaluation Settings . . . . . . . . . . . . . . . 7 | 4. Recommended Evaluation Settings . . . . . . . . . . . . . . . 7 | |||
4.1. Evaluation metircs . . . . . . . . . . . . . . . . . . . 8 | 4.1. Evaluation metircs . . . . . . . . . . . . . . . . . . . 7 | |||
4.2. Path characteristics . . . . . . . . . . . . . . . . . . 8 | 4.2. Path characteristics . . . . . . . . . . . . . . . . . . 8 | |||
4.3. Media source . . . . . . . . . . . . . . . . . . . . . . 9 | 4.3. Media source . . . . . . . . . . . . . . . . . . . . . . 9 | |||
5. Basic Test Cases . . . . . . . . . . . . . . . . . . . . . . 10 | 5. Basic Test Cases . . . . . . . . . . . . . . . . . . . . . . 10 | |||
5.1. Variable Available Capacity with Single RMCAT flow . . . 10 | 5.1. Variable Available Capacity with Single RMCAT flow . . . 10 | |||
5.2. Variable Available Capacity with Multiple RMCAT flows . . 12 | 5.2. Variable Available Capacity with Multiple RMCAT flows . . 12 | |||
5.3. Congested Feedback Link with Bi-directional RMCAT flows . 14 | 5.3. Congested Feedback Link with Bi-directional RMCAT flows . 13 | |||
5.4. Competing Flows with Same RMCAT Algorithm . . . . . . . . 16 | 5.4. Competing Flows with Same RMCAT Algorithm . . . . . . . . 16 | |||
5.5. Round Trip Time Fairness . . . . . . . . . . . . . . . . 18 | 5.5. Round Trip Time Fairness . . . . . . . . . . . . . . . . 18 | |||
5.6. RMCAT Flow competing with a long TCP Flow . . . . . . . . 20 | 5.6. RMCAT Flow competing with a long TCP Flow . . . . . . . . 20 | |||
5.7. RMCAT Flow competing with short TCP Flows . . . . . . . . 22 | 5.7. RMCAT Flow competing with short TCP Flows . . . . . . . . 22 | |||
5.8. Media Pause and Resume . . . . . . . . . . . . . . . . . 24 | 5.8. Media Pause and Resume . . . . . . . . . . . . . . . . . 24 | |||
6. Other potential test cases . . . . . . . . . . . . . . . . . 26 | 6. Other potential test cases . . . . . . . . . . . . . . . . . 26 | |||
6.1. Explicit Congestion Notification Usage . . . . . . . . . 26 | 6.1. Explicit Congestion Notification Usage . . . . . . . . . 26 | |||
6.2. Multiple Bottlenecks . . . . . . . . . . . . . . . . . . 26 | 6.2. Multiple Bottlenecks . . . . . . . . . . . . . . . . . . 26 | |||
7. Wireless Access Links . . . . . . . . . . . . . . . . . . . . 28 | 7. Wireless Access Links . . . . . . . . . . . . . . . . . . . . 28 | |||
7.1. Cellular Network Specific Test Cases . . . . . . . . . . 28 | 7.1. Cellular Network Specific Test Cases . . . . . . . . . . 28 | |||
7.2. Wi-Fi Network Specific Test Cases . . . . . . . . . . . . 28 | 7.2. Wi-Fi Network Specific Test Cases . . . . . . . . . . . . 28 | |||
8. Security Considerations . . . . . . . . . . . . . . . . . . . 29 | 8. Security Considerations . . . . . . . . . . . . . . . . . . . 29 | |||
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 | 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 | |||
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 29 | 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 29 | |||
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 29 | 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 29 | |||
11.1. Normative References . . . . . . . . . . . . . . . . . . 29 | 11.1. Normative References . . . . . . . . . . . . . . . . . . 29 | |||
11.2. Informative References . . . . . . . . . . . . . . . . . 30 | 11.2. Informative References . . . . . . . . . . . . . . . . . 30 | |||
Appendix A. List of Network Parameters . . . . . . . . . . . . . 31 | Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 | |||
A.1. One-way Propagation Delay . . . . . . . . . . . . . . . . 31 | ||||
A.2. End-to-end Loss . . . . . . . . . . . . . . . . . . . . . 31 | ||||
A.3. DropTail Router Queue Length . . . . . . . . . . . . . . 32 | ||||
Appendix B. Models . . . . . . . . . . . . . . . . . . . . . . . 32 | ||||
B.1. Jitter models . . . . . . . . . . . . . . . . . . . . . . 32 | ||||
B.2. Loss generation model . . . . . . . . . . . . . . . . . . 35 | ||||
B.3. TCP taffic model . . . . . . . . . . . . . . . . . . . . 35 | ||||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 35 | ||||
1. Introduction | 1. Introduction | |||
This memo describes a set of test cases for evaluating candidate | This memo describes a set of test cases for evaluating candidate | |||
RMCAT congestion control algorithm proposals, it is based on the | RMCAT congestion control algorithm proposals, it is based on the | |||
guidelines enumerated in [I-D.ietf-rmcat-eval-criteria] and the | guidelines enumerated in [I-D.ietf-rmcat-eval-criteria] and the | |||
requirements discussed in [I-D.ietf-rmcat-cc-requirements]. The test | requirements discussed in [I-D.ietf-rmcat-cc-requirements]. The test | |||
cases cover basic usage scenarios and are described using a common | cases cover basic usage scenarios and are described using a common | |||
structure, which allows for additional test cases to be added to | structure, which allows for additional test cases to be added to | |||
those described herein to accommodate other topologies and/or the | those described herein to accommodate other topologies and/or the | |||
skipping to change at page 5, line 46 | skipping to change at page 5, line 39 | |||
PIE. | PIE. | |||
+ Bottleneck queue size: defines size of queue in terms of | + Bottleneck queue size: defines size of queue in terms of | |||
queuing time when the queue is full (in milliseconds). | queuing time when the queue is full (in milliseconds). | |||
+ Path loss ratio: characterizes the non-congested, additive, | + Path loss ratio: characterizes the non-congested, additive, | |||
losses to be generated on the end-to-end path. MUST | losses to be generated on the end-to-end path. MUST | |||
describe the loss pattern or loss model used to generate the | describe the loss pattern or loss model used to generate the | |||
losses. | losses. | |||
+ Values for some characteristics are described in | ||||
[I-D.ietf-rmcat-eval-criteria]. | ||||
* Application-related: defines the traffic source behaviour for | * Application-related: defines the traffic source behaviour for | |||
implementing the test case | implementing the test case | |||
+ Media traffic Source: defines the characteristics of the | + Media traffic Source: defines the characteristics of the | |||
media sources. When using more than one media source, the | media sources. When using more than one media source, the | |||
different attributes are enumerated separately for each | different attributes are enumerated separately for each | |||
different media source. | different media source. | |||
- Media type: Video/Voice/Application/Text | - Media type: Video/Voice/Application/Text | |||
skipping to change at page 8, line 47 | skipping to change at page 8, line 37 | |||
Each path between a sender and receiver as described in Figure 1 have | Each path between a sender and receiver as described in Figure 1 have | |||
the following characteristics unless otherwise specified in the test | the following characteristics unless otherwise specified in the test | |||
case. | case. | |||
o Path direction: forward and backward. | o Path direction: forward and backward. | |||
o Bottleneck-link capacity: 4Mbps. | o Bottleneck-link capacity: 4Mbps. | |||
o One-Way propagation delay: 50ms. It is encouraged to test with | o One-Way propagation delay: 50ms. It is encouraged to test with | |||
additional propagation delays mentioned in Appendix A.1 | additional propagation delays mentioned in | |||
[I-D.ietf-rmcat-eval-criteria] | ||||
o Maximum end-to-end jitter: 30ms. Jitter models are described in | o Maximum end-to-end jitter: 30ms. Jitter models are described in | |||
Appendix B.1 | [I-D.ietf-rmcat-eval-criteria] | |||
o Bottleneck queue type: Drop tail. It is encouraged to test with | o Bottleneck queue type: Drop tail. It is encouraged to test with | |||
other AQM schemes, such as FQ-CoDel and PIE. | other AQM schemes, such as FQ-CoDel and PIE. | |||
o Bottleneck queue size: 300ms. | o Bottleneck queue size: 300ms. | |||
o Path loss ratio: 0%. | o Path loss ratio: 0%. | |||
Examples of additional network parameters are discussed in | Examples of additional network parameters are discussed in | |||
Appendix A. | [I-D.ietf-rmcat-eval-criteria]. | |||
4.3. Media source | 4.3. Media source | |||
Unless otherwise specified, each test case will include one or more | Unless otherwise specified, each test case will include one or more | |||
media sources as described below. | media sources as described below. | |||
o Media type: Video | o Media type: Video | |||
* Media codec: VBR | * Media codec: VBR | |||
skipping to change at page 16, line 18 | skipping to change at page 16, line 6 | |||
+---------------------+----------------+------------+---------------+ | +---------------------+----------------+------------+---------------+ | |||
| One | Forward | 0s | 2Mbps | | | One | Forward | 0s | 2Mbps | | |||
| Two | Forward | 20s | 1Mbps | | | Two | Forward | 20s | 1Mbps | | |||
| Three | Forward | 40s | 500Kbps | | | Three | Forward | 40s | 500Kbps | | |||
| Four | Forward | 60s | 2Mbps | | | Four | Forward | 60s | 2Mbps | | |||
+---------------------+----------------+------------+---------------+ | +---------------------+----------------+------------+---------------+ | |||
Table 3: Path capacity variation pattern for forward direction | Table 3: Path capacity variation pattern for forward direction | |||
+---------------------+----------------+------------+---------------+ | +---------------------+----------------+------------+---------------+ | |||
| Variation pattern | Path direction | Start time | Path Capacity | | | Variation pattern | Path direction | Start time | Path capacity | | |||
| index | | | | | | index | | | | | |||
+---------------------+----------------+------------+---------------+ | +---------------------+----------------+------------+---------------+ | |||
| One | Backward | 0s | 2Mbps | | | One | Backward | 0s | 2Mbps | | |||
| Two | Backward | 35s | 800Kbps | | | Two | Backward | 35s | 800Kbps | | |||
| Three | Backward | 70s | 2Mbps | | | Three | Backward | 70s | 2Mbps | | |||
+---------------------+----------------+------------+---------------+ | +---------------------+----------------+------------+---------------+ | |||
Table 4: Path capacity variation pattern for backward direction | Table 4: Path capacity variation pattern for backward direction | |||
5.4. Competing Flows with Same RMCAT Algorithm | 5.4. Competing Flows with Same RMCAT Algorithm | |||
skipping to change at page 18, line 15 | skipping to change at page 18, line 7 | |||
- Number of media sources: Three (3) | - Number of media sources: Three (3) | |||
- Media timeline: New media flows are added sequentially, | - Media timeline: New media flows are added sequentially, | |||
at short time intervals. See test specific setup below. | at short time intervals. See test specific setup below. | |||
* Competing traffic: | * Competing traffic: | |||
+ Number of sources : Zero (0) | + Number of sources : Zero (0) | |||
o Test Specific Information: | o Test Specific Information: Table 5 defines the media timeline for | |||
both media type. | ||||
* Media flow timeline: | ||||
+ Flow ID: One (1) | ||||
+ Start time: 0s | ||||
+ End time: 119s | ||||
* Media flow timeline: | ||||
+ Flow ID: Two (2) | ||||
+ Start time: 20s | ||||
+ End time: 119s | ||||
* Media flow timeline: | ||||
+ Flow ID: Three (3) | ||||
+ Start time: 40s | +---------+------------+------------+----------+ | |||
| Flow IF | Media type | Start time | End time | | ||||
+---------+------------+------------+----------+ | ||||
| 1 | Video | 0s | 119s | | ||||
| 2 | Video | 20s | 119s | | ||||
| 3 | Video | 40s | 119s | | ||||
| 4 | Audio | 0s | 119s | | ||||
| 5 | Audio | 20s | 119s | | ||||
| 6 | Audio | 40s | 119s | | ||||
+---------+------------+------------+----------+ | ||||
+ End time: 119s | Table 5: Media Timeline for Video and Audio media sources | |||
5.5. Round Trip Time Fairness | 5.5. Round Trip Time Fairness | |||
In this test case, multiple RMCAT media flows share the bottleneck | In this test case, multiple RMCAT media flows share the bottleneck | |||
link, but the end-to-end path latency for each RMCAT flow is | link, but the end-to-end path latency for each RMCAT flow is | |||
different. For the sake of simplicity it is assumed that there are | different. For the sake of simplicity it is assumed that there are | |||
no other non-RMCAT competing traffic sources in the bottleneck link | no other non-RMCAT competing traffic sources in the bottleneck link | |||
and that there is sufficient capacity to accommodate all the flows. | and that there is sufficient capacity to accommodate all the flows. | |||
While this appears to be a variant of test case 5.2, it focuses on | While this appears to be a variant of test case 5.2, it focuses on | |||
the capacity sharing aspect of the candidate algorithm under | the capacity sharing aspect of the candidate algorithm under | |||
skipping to change at page 19, line 24 | skipping to change at page 19, line 5 | |||
Testbed Topology: Five (5) media sources S1,S2,..,S5 are connected to | Testbed Topology: Five (5) media sources S1,S2,..,S5 are connected to | |||
their corresponding media sinks R1,R2,..,R5. The media traffic is | their corresponding media sinks R1,R2,..,R5. The media traffic is | |||
transported over the forward path and corresponding feedback/control | transported over the forward path and corresponding feedback/control | |||
traffic is transported over the backward path. The topology is the | traffic is transported over the backward path. The topology is the | |||
same as in Section 5.4. The end-to-end path delays are: 10ms for | same as in Section 5.4. The end-to-end path delays are: 10ms for | |||
S1-R1, 25ms for S2-R2, 50ms for S3-R3, 100ms for S4-R4, and 150ms | S1-R1, 25ms for S2-R2, 50ms for S3-R3, 100ms for S4-R4, and 150ms | |||
S5-R5, respectively. | S5-R5, respectively. | |||
Testbed attributes: | Testbed attributes: | |||
o Test duration: 120s | o Test duration: 300s | |||
o Path characteristics: | o Path characteristics: | |||
* One-Way propagation delay for each flow: 10ms, 25ms, 50ms, | * One-Way propagation delay for each flow: 10ms, 25ms, 50ms, | |||
100ms, 150ms. | 100ms, 150ms. | |||
o Application-related: | o Application-related: | |||
* Media Source: | * Media Source: | |||
+ Media type: Video | + Media type: Video | |||
- Media direction: forward | - Media direction: forward | |||
- Number of media sources: Five (5) | - Number of media sources: Five (5) | |||
- Media timeline: | - Media timeline: New media flows are added sequentially, | |||
at short time intervals. See test specific setup below. | ||||
o Start time: 0s. | ||||
o End time: 119s. | ||||
+ Media type: Audio | + Media type: Audio | |||
- Media direction: forward. | - Media direction: forward. | |||
- Number of media sources: Five (5) | - Number of media sources: Five (5) | |||
- Media timeline: | ||||
o Start time: 0s. | - Media timeline: New media flows are added sequentially, | |||
at short time intervals. See test specific setup below. | ||||
o End time: 119s. | ||||
* Competing traffic: | * Competing traffic: | |||
+ Number of sources : Zero (0) | + Number of sources : Zero (0) | |||
o Test Specific Information: None | o Test Specific Information: Table 6 defines the media timeline for | |||
both media type. | ||||
+---------+------------+------------+----------+ | ||||
| Flow IF | Media type | Start time | End time | | ||||
+---------+------------+------------+----------+ | ||||
| 1 | Video | 0s | 299s | | ||||
| 2 | Video | 10s | 299s | | ||||
| 3 | Video | 20s | 299s | | ||||
| 4 | Video | 30s | 299s | | ||||
| 5 | Video | 40s | 299s | | ||||
| 6 | Audio | 0 | 299s | | ||||
| 7 | Audio | 10s | 299s | | ||||
| 8 | Audio | 20s | 299s | | ||||
| 9 | Audio | 30s | 299s | | ||||
| 10 | Audio | 40s | 299s | | ||||
+---------+------------+------------+----------+ | ||||
Table 6: Media Timeline for Video and Audio media sources | ||||
5.6. RMCAT Flow competing with a long TCP Flow | 5.6. RMCAT Flow competing with a long TCP Flow | |||
In this test case, one or more RMCAT media flows share the bottleneck | In this test case, one or more RMCAT media flows share the bottleneck | |||
link with at least one long lived TCP flows. Long lived TCP flows | link with at least one long lived TCP flows. Long lived TCP flows | |||
download data throughout the session and are expected to have | download data throughout the session and are expected to have | |||
infinite amount of data to send and receive. This is a scenario | infinite amount of data to send and receive. This is a scenario | |||
where a multimedia application co-exists with a large file download. | where a multimedia application co-exists with a large file download. | |||
The test case measures the adaptivity of the candidate algorithm to | The test case measures the adaptivity of the candidate algorithm to | |||
competing traffic. It addresses the requirement 3 in | competing traffic. It addresses the requirement 3 in | |||
skipping to change at page 30, line 8 | skipping to change at page 30, line 8 | |||
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July | Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July | |||
2006. | 2006. | |||
[RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size | [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size | |||
Real-Time Transport Control Protocol (RTCP): Opportunities | Real-Time Transport Control Protocol (RTCP): Opportunities | |||
and Consequences", RFC 5506, April 2009. | and Consequences", RFC 5506, April 2009. | |||
[I-D.ietf-avtcore-rtp-circuit-breakers] | [I-D.ietf-avtcore-rtp-circuit-breakers] | |||
Perkins, C. and V. Singh, "Multimedia Congestion Control: | Perkins, C. and V. Singh, "Multimedia Congestion Control: | |||
Circuit Breakers for Unicast RTP Sessions", draft-ietf- | Circuit Breakers for Unicast RTP Sessions", draft-ietf- | |||
avtcore-rtp-circuit-breakers-05 (work in progress), | avtcore-rtp-circuit-breakers-09 (work in progress), March | |||
February 2014. | 2015. | |||
[I-D.ietf-rmcat-eval-criteria] | [I-D.ietf-rmcat-eval-criteria] | |||
Singh, V. and J. Ott, "Evaluating Congestion Control for | Singh, V. and J. Ott, "Evaluating Congestion Control for | |||
Interactive Real-time Media", draft-ietf-rmcat-eval- | Interactive Real-time Media", draft-ietf-rmcat-eval- | |||
criteria-00 (work in progress), January 2014. | criteria-02 (work in progress), July 2014. | |||
[I-D.ietf-rmcat-cc-requirements] | [I-D.ietf-rmcat-cc-requirements] | |||
Jesup, R., "Congestion Control Requirements For RMCAT", | Jesup, R. and Z. Sarker, "Congestion Control Requirements | |||
draft-ietf-rmcat-cc-requirements-02 (work in progress), | for Interactive Real-Time Media", draft-ietf-rmcat-cc- | |||
February 2014. | requirements-09 (work in progress), December 2014. | |||
[I-D.draft-sarker-rmcat-cellular-eval-test-cases] | [I-D.draft-sarker-rmcat-cellular-eval-test-cases] | |||
Sarker, Z. and I. Johansson, "Evaluation Test Cases for | Sarker, Z. and I. Johansson, "Evaluation Test Cases for | |||
Interactive Real-Time Media over Cellular Networks", 6 | Interactive Real-Time Media over Cellular Networks", 6 | |||
2014, <http://www.ietf.org/id/ | 2014, <http://www.ietf.org/id/ | |||
draft-sarker-rmcat-cellular-eval-test-cases-01.txt>. | draft-sarker-rmcat-cellular-eval-test-cases-01.txt>. | |||
11.2. Informative References | 11.2. Informative References | |||
[I-D.ietf-rtcweb-use-cases-and-requirements] | ||||
Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real- | ||||
Time Communication Use-cases and Requirements", draft- | ||||
ietf-rtcweb-use-cases-and-requirements-14 (work in | ||||
progress), February 2014. | ||||
[RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion | ||||
Control Algorithms", BCP 133, RFC 5033, August 2007. | ||||
[RFC5166] Floyd, S., "Metrics for the Evaluation of Congestion | ||||
Control Mechanisms", RFC 5166, March 2008. | ||||
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion | ||||
Control", RFC 5681, September 2009. | ||||
[SA4-EVAL] | ||||
R1-081955, 3GPP., "LTE Link Level Throughput Data for SA4 | ||||
Evaluation Framework", 3GPP R1-081955, 5 2008. | ||||
[SA4-LR] S4-050560, 3GPP., "Error Patterns for MBMS Streaming over | ||||
UTRAN and GERAN", 3GPP S4-050560, 5 2008. | ||||
[xiph-seq] | [xiph-seq] | |||
Xiph.org, , "Video Test Media", | Xiph.org, , "Video Test Media", | |||
http://media.xiph.org/video/derf/ , . | http://media.xiph.org/video/derf/ , . | |||
[HEVC-seq] | [HEVC-seq] | |||
HEVC, , "Test Sequences", | HEVC, , "Test Sequences", | |||
http://www.netlab.tkk.fi/~varun/test_sequences/ , . | http://www.netlab.tkk.fi/~varun/test_sequences/ , . | |||
[TCP-eval-suite] | ||||
Lachlan, A., Marcondes, C., Floyd, S., Dunn, L., Guillier, | ||||
R., Gang, W., Eggert, L., Ha, S., and I. Rhee, "Towards a | ||||
Common TCP Evaluation Suite", Proc. PFLDnet. 2008, August | ||||
2008. | ||||
Appendix A. List of Network Parameters | ||||
In addition to the recommended evaluation settings in Section 4, the | ||||
implemntors can extend their tests by choosing from the following | ||||
values: | ||||
A.1. One-way Propagation Delay | ||||
Experiments are expected to verify that the congestion control is | ||||
able to work in challenging situations, for example over trans- | ||||
continental and/or satellite links. Typical values are: | ||||
1. Very low latency: 0-1ms | ||||
2. Low latency: 50ms | ||||
3. High latency: 150ms | ||||
4. Extreme latency: 300ms | ||||
A.2. End-to-end Loss | ||||
To model lossy links, the experiments can choose one of the following | ||||
loss rates, the fractional loss is the ratio of packets lost and | ||||
packets sent. | ||||
1. no loss: 0% | ||||
2. 1% | ||||
3. 5% | ||||
4. 10% | ||||
5. 20% | ||||
A.3. DropTail Router Queue Length | ||||
The router queue length is measured as the time taken to drain the | ||||
FIFO queue. It has been noted in various discussions that the queue | ||||
length in the current deployed Internet varies significantly. While | ||||
the core backbone network has very short queue length, the home | ||||
gateways usually have larger queue length. Those various queue | ||||
lengths can be categorized in the following way: | ||||
1. QoS-aware (or short): 70ms | ||||
2. Nominal: 300-500ms | ||||
3. Buffer-bloated: 1000-2000ms | ||||
Here the size of the queue is measured in bytes or packets and to | ||||
convert the queue length measured in seconds to queue length in | ||||
bytes: | ||||
QueueSize (in bytes) = QueueSize (in sec) x Throughput (in bps)/8 | ||||
Appendix B. Models | ||||
B.1. Jitter models | ||||
This section defines jitter model for the purposes of this document. | ||||
When jitter is to be applied to both the RMCAT flow and any competing | ||||
flow (such as a TCP competing flow), the competing flow will use the | ||||
jitter definition below that does not allow for re-ordering of | ||||
packets on the competing flow (see NR-RBPDV definition below). | ||||
Jitter is an overloaded term in communications. Its meaning is | ||||
typically associated with the variation of a metric (e.g., delay) | ||||
with respect to some reference metric (e.g., average delay or minimum | ||||
delay). For example, RFC 3550 jitter is a smoothed estimate of | ||||
jitter which is particularly meaningful if the underlying packet | ||||
delay variation was caused by a Gaussian random process. | ||||
Because jitter is an overloaded term, we instead use the term Packet | ||||
Delay Variation (PDV) to describe the variation of delay of | ||||
individual packets in the same sense as the IETF IPPM WG has defined | ||||
PDV in their documents (e.g., RFC 3393) and as the ITU-T SG16 has | ||||
defined IP Packet Delay Variation (IPDV) in their documents (e.g., | ||||
Y.1540). | ||||
Most PDV distributions in packet network systems are one-sided | ||||
distributions (the measurement of which with a finite number of | ||||
measurement samples result in one-sided histograms). In the usual | ||||
packet network transport case there is typically one packet that | ||||
transited the network with the minimum delay, then a majority of | ||||
packets also transit the system within some variation from this | ||||
minimum delay, and then a minority of the packets transits the | ||||
network with delays higher than the median or average transit time | ||||
(these are outliers). Although infrequent, outliers can cause | ||||
significant deleterious operation in adaptive systems and should be | ||||
considered in RMCAT adaptation designs. | ||||
In this section we define two different bounded PDV characteristics, | ||||
1) Random Bounded PDV and 2) Approximately Random Subject to No- | ||||
Reordering Bounded PDV. | ||||
Random Bounded PDV (RBPDV): | ||||
The RBPDV probability distribution function (pdf) is specified to be | ||||
of some mathematically describable function which includes some | ||||
practical minimum and maximum discrete values suitable for testing. | ||||
For example, the minimum value, x_min, might be specified as the | ||||
minimum transit time packet and the maximum value, x_max, might be | ||||
idefined to be two standard deviations higher than the mean. | ||||
Since we are typically interested in the distribution relative to the | ||||
mean delay packet, we define the zero mean PVD sample, z(n), to be | ||||
z(n) = x(n) - x_mean, where x(n) is a sample of the RBPDV random | ||||
variable x and x_mean is the mean of x. | ||||
We assume here that s(n) is the original source time of packet n and | ||||
the post-jitter induced emmission time, j(n), for packet n is j(n) = | ||||
{[z(n) + x_mean] + s(n)}. It follows that the separation in the post- | ||||
jitter time of packets n and n+1 is {[s(n+1)-s(n)] - [z(n)-z(n+1)]}. | ||||
Since the first term is always a positive quantity, we note that | ||||
packet reordering at the receiver is possible whenever the second | ||||
term is greater than the first. Said another way, whenever the | ||||
difference in possible zero mean PDV sample delays (i.e., [x_max- | ||||
x_min]) exceeds the inter-departure time of any two sent packets, we | ||||
have the possibility of packet re-ordering. | ||||
There are important use cases in real networks where packets can | ||||
become re-ordered such as in load balancing topologies and during | ||||
route changes. However, for the vast majority of cases there is no | ||||
packet re-ordering because most of the time packets follow the same | ||||
path. Due to this, if a packet becomes overly delayed, the packets | ||||
after it on that flow are also delayed. This is especially true for | ||||
mobile wireless links where there are per-flow queues prior to base | ||||
station scheduling. Owing to this important use case, we define | ||||
another PDV profile similar to the above, but one that does not allow | ||||
for re-ordering within a flow. | ||||
Approximately Random Subject to No-Reordering Bounded PDV (NR-RPVD): | ||||
No Reordering RPDV, NR-RPVD, is defined similarly to the above with | ||||
one important exception. Let serial(n) be defined as the | ||||
serialization delay of packet n at the lowest bottleneck link rate | ||||
(or other appropriate rate) in a given test. Then we produce all the | ||||
post-jitter values for j(n) for n = 1, 2, ... N, where N is the | ||||
length of the source sequence s to be jittered. The exception can be | ||||
stated as follows: We revisit all j(n) beginning from index n=2, and | ||||
if j(n) is determined to be less than [j(n-1)+serial(n-1)], we | ||||
redefine j(n) to be equal to [j(n-1)+serial(n-1)] and continue for | ||||
all remaining n (i.e., n = 3, 4, .. N). This models the case where | ||||
the packet n is sent immediately after packet (n-1) at the bottleneck | ||||
link rate. Although this is generally the theoretical minimum in | ||||
that it assumes that no other packets from other flows are in-between | ||||
packet n and n+1 at the bottleneck link, it is a reasonable | ||||
assumption for per flow queuing. | ||||
We note that this assumption holds for some important exception | ||||
cases, such as packets immediately following outliers. There are a | ||||
multitude of software controlled elements common on end-to-end | ||||
Internet paths (such as firewalls, ALGs and other middleboxes) which | ||||
stop processing packets while servicing other functions (e.g., | ||||
garbage collection). Often these devices do not drop packets, but | ||||
rather queue them for later processing and cause many of the | ||||
outliers. Thus NR-RPVD models this particular use case (assuming | ||||
serial(n+1) is defined appropriately for the device causing the | ||||
outlier) and thus is believed to be important for adaptation | ||||
development for RMCAT. | ||||
[Editor's Note: It may require to define test distributions as well. | ||||
Example test distrubution may include- | ||||
1 - Two-sided: Uniform PDV Distribution. Two quantities to define: | ||||
x_min and x_max. | ||||
2 - Two-sided: Truncated Gaussian PDV Distribution. Four quantities | ||||
to define: the appropriate x_min and x_max for test (e.g., +/- two | ||||
sigma values), the standard deviation and the mean. | ||||
3 - One Sided: TBD] | ||||
B.2. Loss generation model | ||||
[Editor's note : Describes the model for generating packet losses, | ||||
for example, losses can be generated using traces, or using the | ||||
Gilbert-Elliot model, or randomly (uncorrelated loss).] | ||||
B.3. TCP taffic model | ||||
Long-lived TCP flows will download data throughout the session and | ||||
are expected to have infinite amount of data to send or receive. | ||||
Each short TCP flow is modeled as a sequence of file downloads | ||||
interleaved with idle periods. Not all short TCPs start at the same | ||||
time, i.e., some start in the ON state while others start in the OFF | ||||
state. | ||||
The short TCP flows can be modelled in two ways, 1) 100s of flows | ||||
fetching small (5-20 KB) amounts of data, or 2) 10s of flows fetching | ||||
slightly larger (100-1000KB) amounts of data. | ||||
The idle period is typically derived from an exponential distribution | ||||
with the mean value of 10 seconds. | ||||
[Open issue: short-lived/bursty TCP cross-traffic parameters are | ||||
still to be agreed upon]. | ||||
Authors' Addresses | Authors' Addresses | |||
Zaheduzzaman Sarker | Zaheduzzaman Sarker | |||
Ericsson AB | Ericsson AB | |||
Luleae, SE 977 53 | Luleae, SE 977 53 | |||
Sweden | Sweden | |||
Phone: +46 10 717 37 43 | Phone: +46 10 717 37 43 | |||
Email: zaheduzzaman.sarker@ericsson.com | Email: zaheduzzaman.sarker@ericsson.com | |||
Varun Singh | Varun Singh | |||
Aalto University | Aalto University | |||
School of Electrical Engineering | School of Electrical Engineering | |||
Otakaari 5 A | Otakaari 5 A | |||
Espoo, FIN 02150 | Espoo, FIN 02150 | |||
Finland | Finland | |||
Email: varun@comnet.tkk.fi | Email: varun@comnet.tkk.fi | |||
URI: http://www.netlab.tkk.fi/~varun/ | URI: http://www.netlab.tkk.fi/~varun/ | |||
Xiaoqing Zhu | Xiaoqing Zhu | |||
Cisco Systems | Cisco Systems | |||
510 McCarthy Blvd | 510 McCarthy Blvd | |||
Milpitas, CA 95134 | Milpitas, CA 95134 | |||
USA | USA | |||
Email: xiaoqzhu@cisco.com | Email: xiaoqzhu@cisco.com | |||
Michael A. Ramalho | Michael A. Ramalho | |||
Cisco Systems, Inc. | Cisco Systems, Inc. | |||
skipping to change at page 36, line 14 | skipping to change at page 31, line 24 | |||
Xiaoqing Zhu | Xiaoqing Zhu | |||
Cisco Systems | Cisco Systems | |||
510 McCarthy Blvd | 510 McCarthy Blvd | |||
Milpitas, CA 95134 | Milpitas, CA 95134 | |||
USA | USA | |||
Email: xiaoqzhu@cisco.com | Email: xiaoqzhu@cisco.com | |||
Michael A. Ramalho | Michael A. Ramalho | |||
Cisco Systems, Inc. | Cisco Systems, Inc. | |||
8000 Hawkins Road | 6310 Watercrest Way Unit 203 | |||
Sarasota, FL 34241 | Lakewood Ranch, FL 34202-5211 | |||
USA | USA | |||
Phone: +1 919 476 2038 | Phone: +1 919 476 2038 | |||
Email: mramalho@cisco.com | Email: mramalho@cisco.com | |||
End of changes. 30 change blocks. | ||||
292 lines changed or deleted | 63 lines changed or added | |||
This html diff was produced by rfcdiff 1.42. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |