Abstract
Understanding the performance of the Internet's Transmission Control Protocol (TCP) is important because it is the dominant protocol used in the Internet today. Various testing methods exist to evaluate TCP performance, however all have pitfalls that need to be understood prior to obtaining useful results. Simulating TCP is difficult because of the wide range of variables, environments, and implementations available. Testing TCP modifications in the global Internet may not be the answer either: testing new protocols on real networks endangers other people's traffic and, if not done correctly, may also yield inaccurate or misleading results. In order for TCP research to be independently evaluated in the Internet research community there is a set of questions that researchers should try to answer. This paper attempts to list some of those questions and make recommendations as to how TCP testing can be structured to provide useful answers.
- {ADLY95} Jong Suk Ahn, Peter Danzig, Zhen Liu, and Limin Yan. Evaluation of TCP Vegas: Emulation and Experiment. In ACM SIGCOMM, 1995.]] Google ScholarDigital Library
- {AFP98} Mark Allman, Sally Floyd, and Craig Partridge. Increasing TCP's Initial Window, September 1998. RFC 2414.]]Google Scholar
- {AGS99} Mark Allman, Dan Glover, and Luis Sanchez. Enhancing TCP Over Satellite Channels Using Standard Mechanisms, January 1999. RFC 2488.]]Google Scholar
- {A1197a} Mark Allman. Fixing Two BSD TCP Bugs. Technical Report CR-204151, NASA Lewis Research Center, October 1997.]]Google Scholar
- {All97b} Mark Allman. Improving TCP Performance Over Satellite Channels. Master's thesis, Ohio University, June 1997.]]Google Scholar
- {All98} Mark Allman. On the Generation and Use of TCP Acknowledgments. Computer Communication Review, 28(5), October 1998.]] Google ScholarDigital Library
- {All99} Mark Allman. TCP Byte Counting Refinements. Computer Communication Review, 29(3), July 1999.]] Google ScholarDigital Library
- {AP99} Mark Allman and Vern Paxson. On Estimating End-to-End Network Path Properties. In ACM SIGCOMM, September 1999. To appear.]] Google ScholarDigital Library
- {APS99} Mark Allman, Vern Paxson, and W. Richard Stevens. TCP Congestion Control, April 1999. RFC 2581.]]Google Scholar
- {Arc} Internet Traffic Archive. http://ita.ee.lbl.gov.]]Google Scholar
- {BCC+98} Robert Braden, David Clark, Jon Crowcroft, Bruce Davie, Steve Deering, Deborah Estrin, Sally Floyd, Van Jacobson, Greg Minshall, Craig Partridge, Larry Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski, and Lixia Zhang. Recommendations on Queue Management and Congestion Avoidance in the Internet, April 1998. RFC 2309.]]Google Scholar
- {BLFN96} Tim Berners-Lee, R. Fielding, and H. Nielsen Hypertext Transfer Protocol --- HTTP/1.0, May 1996. RFC 1945.]]Google Scholar
- {BP95} Lawrence Brakmo and Larry Peterson. TCP Vegas: End to End Congestion Avoidance on a Global Internet. IEEE Journal on Selected Areas in Communications, 13(8), October 1995.]]Google Scholar
- {BP96} Lawrence Brakmo and Larry Peterson. Experiences with Network Simulators. In ACM SIGMETRICS, 1996.]] Google ScholarDigital Library
- {Bra89} Robert Braden. Requirements for Internet Hosts --- Communication Layers, October 1989. RFC 1122.]]Google Scholar
- {DJ91} Peter Danzig and Sugih Jamin. tcplib: A Library of TCP/IP Traffic Characteristics. Technical Report CS-SYS-91-01, University of Southern California, October 1991.]]Google Scholar
- {Fal99} Kevin Fall. Network Emulation in the Vint/NS Simulator. Technical report, University of California, Berkeley, January 1999.]]Google Scholar
- {Fey85} Richard Feynman. Surely You're Joking Mr. Feynman!, chapter Cargo Cult Science, pages 308-317. Bantam, 1985.]]Google Scholar
- {FF96} Kevin Fall and Sally Floyd. Simulation-based Comparisons of Tahoe, Reno, and SACK TCP. Computer Communications Review, 26(3), July 1996.]] Google ScholarDigital Library
- {FGM+97} R. Fielding, Jim Gettys, Jeffrey C. Mogul, H. Frystyk, and Tim Berners-Lee. Hypertext Transfer Protocol --- HTTP/1.1, January 1997. RFC 2068.]] Google ScholarDigital Library
- {FJ93} Sally Floyd and Van Jacobson. Random Early Detection Gateways for Congestion Avoidance. IEEE/ACM Transactions on Networking, 1(4):397-413, August 1993.]] Google ScholarDigital Library
- {Flo94} Sally Floyd. TCP and Explicit Congestion Notification. Computer Communications Review, 24(5):10-23, October 1994.]] Google ScholarDigital Library
- {Flo99} Sally Floyd. Validation Experiences with the NS Simulator. Technical report, ACIRI, April 1999. White paper submitted to the DARPA Workshop on Validation of Large Scale Network Simulation Models.]]Google Scholar
- {Hei97} John Heidemann. Performance Interactions Between P-HTTP and TCP Implementations. Computer Communication Review, 27(2):65-73, April 1997.]] Google ScholarDigital Library
- {Hey90} A. Heybey. The Network Simulator. Technical report, MIT, September 1990.]]Google Scholar
- {HK99} Tom Henderson and Randy Katz. Transport Protocols for Internet-Compatible Satellite Networks. IEEE Journal on Selected Areas of Communications, 1999. To appear.]]Google ScholarDigital Library
- {Hoe96} Janey Hoe. Improving the Start-up Behavior of a Congestion Control Scheme for TCP. In ACM SIGCOMM, August 1996.]] Google ScholarDigital Library
- {Jac88} Van Jacobson. Congestion Avoidance and Control. In ACM SIGCOMM, 1988.]] Google ScholarDigital Library
- {Jai91} Raj Jain. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation and Modeling. Wiley, 1991.]]Google Scholar
- {JBB92} Van Jacobson, Robert Braden, and David Borman. TCP Extensions for High Performance, May 1992. RFC 1323.]] Google ScholarDigital Library
- {Kes88} Srinivasan Keshav. REAL: A Network Simulator. Technical Report 88/472, University of California Berkeley, 1988.]] Google ScholarDigital Library
- {KL86} B. Kantor and P. Lapsley. Network News Transfer Protocol: A Proposed Standard for the Stream-Based Transmission of News, February 1986. RFC 977.]] Google ScholarDigital Library
- {MA99} Matt Mathis and Mark Allman. Empirical Bulk Transfer Capacity, June 1999. Internet-Draft draft-ietf-ippm-btc-framework-01.txt.]]Google Scholar
- {Mah99} Jamshid Mahdavi. Enabling High Performance Data Transfers on Hosts. Technical report, Pittsburgh Supercomputer Center, June 1999. http://www.psc.edu/networking/perf_tune.html.]]Google Scholar
- {MD90} Jeffrey C. Mogul and Steve Deering. Path MTU Discovery, November 1990. RFC 1191.]]Google ScholarDigital Library
- {MDM96} Jack McCann, Steve Deering, and Jeffrey C. Mogul. Path MTU Discovery for IP version 6, August 1996. RFC 1981.]]Google ScholarDigital Library
- {MF95} Steven McCanne and Sally Floyd. NS (Network Simulator), 1995. URL http://www-nrg.ee.lbl.gov.]]Google Scholar
- {MM96a} Matt Mathis and Jamshid Mahdavi. Diagnosing Internet Congestion with a Transport Layer Performance Tool. In Proceedings of INET, June 1996.]]Google Scholar
- {MM96b} Matt Mathis and Jamshid Mahdavi. Forward Acknowledgment: Refining TCP Congestion Control. In ACM SIGCOMM, August -1996.]] Google ScholarDigital Library
- {MM96c} Matt Mathis and Jamshid Mahdavi. TCP Rate-Halving with Bounding Parameters. Technical report, Pittsburgh Supercomputer Center, October 1996. URL: http://www.psc.edu/networking/papers/FACKnotes/current/.]]Google Scholar
- {MMFR96} Matt Mathis, Jamshid Mahdavi, Sally Floyd, and Allyn Romanow. TCP Selective Acknowledgement Options, October 1996. RFC 2018.]]Google ScholarDigital Library
- {Mog92} Jeffrey C. Mogul. Observing TCP Dynamics in Real Networks. In ACM SIGCOMM, pages 305-317, 1992.]] Google ScholarDigital Library
- {MSML99} Matt Mathis, Jeff Semke, Jamshid Mahdavi, and Kevin Lahey. The Rate-Halving Algorithm for TCP Congestion Control, August 1999. Internet-Draft draft-mathis-tcp-ratehalving-00.txt (work in progress).]]Google Scholar
- {MSMO97} Matt Mathis, Jeff Semke, Jamshid Mahdavi, and Teunis Ott. The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm. Computer Communication Review, 27(3), July 1997.]] Google ScholarDigital Library
- {MSMV99} Greg Minshall, Yasushi Saito, Jeffrey C. Mogul, and Ben Verghese. Application Performance Pitfalls and TCP's Nagle Algorithm. In Workshop on Internet Server Performance, May 1999.]]Google Scholar
- {Nag84} John Nagle. Congestion Control in IP/TCP Internetworks, January 1984. RFC 896.]]Google Scholar
- {PAD+99} Vern Paxson, Mark Allman, Scott Dawson, William Fenner, Jim Griner, Ian Heavens, Kevin Lahey, Jeff Semke, and Bernie Volz. Known TCP Implementation Problems, March 1999. RFC 2525.]]Google Scholar
- {Pax97a} Vern Paxson. Automated Packet Trace Analysis of TCP Implementations. In ACM SIGCOMM, September 1997.]] Google ScholarDigital Library
- {Pax97b} Vern Paxson. End-to-End Internet Packet Dynamics. In ACM SIGCOMM, September 1997.]] Google ScholarDigital Library
- {Pax97c} Vern Paxson. Measurements and Analysis of End-to-End Internet Dynamics. Ph.D. thesis, University of California Berkeley, 1997.]]Google Scholar
- {PBS+96} Venkata Padmanabhan, Hari Balakrishnan, Keith Sklower, Elan Amir, and Randy Katz. Networking Using Direct Broadcast Saellite. In Proceedings of the First International Workshop on Satellite-based Information Services (WOSBIS), November 1996.]]Google Scholar
- {PF95} Vern Paxson and Sally Floyd. Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, 3(3):226-244, June 1995.]] Google ScholarDigital Library
- {PF97} Vern Paxson and Sally Floyd. Why We Don't Know How to Simulate the Internet. In Proceedings of the 1997 Winter Simulation Conference,December 1997.]] Google ScholarDigital Library
- {PMAM98} Vern Paxson, Jamshid Mahdavi, Andrew Adams, and Matt Mathis. An Architecture for Large-Scale Internet Measurement. IEEE Communications, 1998.]]Google ScholarDigital Library
- {Pos81} Jon Postel. Transmission Control Protocol, September 1981. RFC 793.]]Google Scholar
- {PR85} Jon Postel and Joyce Reynolds. File Tranfer Protocol (FTP), October 1985. RFC 959.]]Google Scholar
- {PS98} Steve Parker and Chris Schmechel. Some Testing Tools for TCP Implementors, August 1998. RFC 2398.]]Google Scholar
- {RF99} K. K. Ramakrishnan and Sally Floyd. A Proposal to Add Explicit Congestion Notification (ECN) to IP, January 1999. RFC 2481.]] Google ScholarDigital Library
- {Ric99} Alan Richard. ISINT Performance Validation Test Report. Technical Report CR-209407, NASA Glenn Research Center, June 1999.]]Google Scholar
- {Riz97} Luigi Rizzo. Dummynet: A Simple Approach to the Evaluation of Network Protocols. Computer Communications Review, 27(1):31-41, January 1997.]] Google ScholarDigital Library
- {SMM98} Jeff Semke, Jamshid Mahdavi, and Matt Mathis. Automatic TCP Buffer Tuning. In ACM SIGCOMM, September 1998.]] Google ScholarDigital Library
- {Tec} Third Millennium Technologies. OpNet Modeler. http://www.mil3.com.]]Google Scholar
- {TMW97} Kevin Thompson, Gregory Miller, and Rick Wilder. Wide-Area Internet Traffic Patterns and Characteristics. IEEE Network, 11(6):10-23, November/December 1997.]]Google ScholarDigital Library
- {VH97} Vikram Visweswaraiah and John Heidemann. Improving Restart of Idle TCP Connections. Technical Report 97-661, University of Southern California, August 1997.]]Google Scholar
Index Terms
- On the effective evaluation of TCP
Recommendations
Performance evaluation of Westwood+ TCP congestion control
Internet performance symposium (IPS 2002)Westwood+ TCP is a sender-side only modification of the classic Tahoe/Reno TCP that has been recently proposed to improve fairness and efficiency of TCP. The key idea of Westwood+ TCP is to perform an end-to-end estimate of the bandwidth available for a ...
TCP CERL: congestion control enhancement over wireless networks
In this paper, we propose and verify a modified version of TCP Reno that we call TCP Congestion Control Enhancement for Random Loss (CERL). We compare the performance of TCP CERL, using simulations conducted in ns-2, to the following other TCP variants: ...
TCP Prairie: a sender-only TCP modification based on adaptive bandwidth estimation in wired-wireless networks
This paper presents a new TCP (Transmission Control Protocol) variant called TCP Prairie which is a sender-only TCP modification based on dynamic bandwidth estimation in wired-cum-wireless networks. The key idea here is to continuously measure the ...
Comments