skip to main content
article
Free Access

On the effective evaluation of TCP

Published:05 October 1999Publication History
Skip Abstract Section

Abstract

Understanding the performance of the Internet's Transmission Control Protocol (TCP) is important because it is the dominant protocol used in the Internet today. Various testing methods exist to evaluate TCP performance, however all have pitfalls that need to be understood prior to obtaining useful results. Simulating TCP is difficult because of the wide range of variables, environments, and implementations available. Testing TCP modifications in the global Internet may not be the answer either: testing new protocols on real networks endangers other people's traffic and, if not done correctly, may also yield inaccurate or misleading results. In order for TCP research to be independently evaluated in the Internet research community there is a set of questions that researchers should try to answer. This paper attempts to list some of those questions and make recommendations as to how TCP testing can be structured to provide useful answers.

References

  1. {ADLY95} Jong Suk Ahn, Peter Danzig, Zhen Liu, and Limin Yan. Evaluation of TCP Vegas: Emulation and Experiment. In ACM SIGCOMM, 1995.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. {AFP98} Mark Allman, Sally Floyd, and Craig Partridge. Increasing TCP's Initial Window, September 1998. RFC 2414.]]Google ScholarGoogle Scholar
  3. {AGS99} Mark Allman, Dan Glover, and Luis Sanchez. Enhancing TCP Over Satellite Channels Using Standard Mechanisms, January 1999. RFC 2488.]]Google ScholarGoogle Scholar
  4. {A1197a} Mark Allman. Fixing Two BSD TCP Bugs. Technical Report CR-204151, NASA Lewis Research Center, October 1997.]]Google ScholarGoogle Scholar
  5. {All97b} Mark Allman. Improving TCP Performance Over Satellite Channels. Master's thesis, Ohio University, June 1997.]]Google ScholarGoogle Scholar
  6. {All98} Mark Allman. On the Generation and Use of TCP Acknowledgments. Computer Communication Review, 28(5), October 1998.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. {All99} Mark Allman. TCP Byte Counting Refinements. Computer Communication Review, 29(3), July 1999.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. {AP99} Mark Allman and Vern Paxson. On Estimating End-to-End Network Path Properties. In ACM SIGCOMM, September 1999. To appear.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. {APS99} Mark Allman, Vern Paxson, and W. Richard Stevens. TCP Congestion Control, April 1999. RFC 2581.]]Google ScholarGoogle Scholar
  10. {Arc} Internet Traffic Archive. http://ita.ee.lbl.gov.]]Google ScholarGoogle Scholar
  11. {BCC+98} Robert Braden, David Clark, Jon Crowcroft, Bruce Davie, Steve Deering, Deborah Estrin, Sally Floyd, Van Jacobson, Greg Minshall, Craig Partridge, Larry Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski, and Lixia Zhang. Recommendations on Queue Management and Congestion Avoidance in the Internet, April 1998. RFC 2309.]]Google ScholarGoogle Scholar
  12. {BLFN96} Tim Berners-Lee, R. Fielding, and H. Nielsen Hypertext Transfer Protocol --- HTTP/1.0, May 1996. RFC 1945.]]Google ScholarGoogle Scholar
  13. {BP95} Lawrence Brakmo and Larry Peterson. TCP Vegas: End to End Congestion Avoidance on a Global Internet. IEEE Journal on Selected Areas in Communications, 13(8), October 1995.]]Google ScholarGoogle Scholar
  14. {BP96} Lawrence Brakmo and Larry Peterson. Experiences with Network Simulators. In ACM SIGMETRICS, 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. {Bra89} Robert Braden. Requirements for Internet Hosts --- Communication Layers, October 1989. RFC 1122.]]Google ScholarGoogle Scholar
  16. {DJ91} Peter Danzig and Sugih Jamin. tcplib: A Library of TCP/IP Traffic Characteristics. Technical Report CS-SYS-91-01, University of Southern California, October 1991.]]Google ScholarGoogle Scholar
  17. {Fal99} Kevin Fall. Network Emulation in the Vint/NS Simulator. Technical report, University of California, Berkeley, January 1999.]]Google ScholarGoogle Scholar
  18. {Fey85} Richard Feynman. Surely You're Joking Mr. Feynman!, chapter Cargo Cult Science, pages 308-317. Bantam, 1985.]]Google ScholarGoogle Scholar
  19. {FF96} Kevin Fall and Sally Floyd. Simulation-based Comparisons of Tahoe, Reno, and SACK TCP. Computer Communications Review, 26(3), July 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. {FGM+97} R. Fielding, Jim Gettys, Jeffrey C. Mogul, H. Frystyk, and Tim Berners-Lee. Hypertext Transfer Protocol --- HTTP/1.1, January 1997. RFC 2068.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. {FJ93} Sally Floyd and Van Jacobson. Random Early Detection Gateways for Congestion Avoidance. IEEE/ACM Transactions on Networking, 1(4):397-413, August 1993.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. {Flo94} Sally Floyd. TCP and Explicit Congestion Notification. Computer Communications Review, 24(5):10-23, October 1994.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. {Flo99} Sally Floyd. Validation Experiences with the NS Simulator. Technical report, ACIRI, April 1999. White paper submitted to the DARPA Workshop on Validation of Large Scale Network Simulation Models.]]Google ScholarGoogle Scholar
  24. {Hei97} John Heidemann. Performance Interactions Between P-HTTP and TCP Implementations. Computer Communication Review, 27(2):65-73, April 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. {Hey90} A. Heybey. The Network Simulator. Technical report, MIT, September 1990.]]Google ScholarGoogle Scholar
  26. {HK99} Tom Henderson and Randy Katz. Transport Protocols for Internet-Compatible Satellite Networks. IEEE Journal on Selected Areas of Communications, 1999. To appear.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. {Hoe96} Janey Hoe. Improving the Start-up Behavior of a Congestion Control Scheme for TCP. In ACM SIGCOMM, August 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. {Jac88} Van Jacobson. Congestion Avoidance and Control. In ACM SIGCOMM, 1988.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. {Jai91} Raj Jain. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation and Modeling. Wiley, 1991.]]Google ScholarGoogle Scholar
  30. {JBB92} Van Jacobson, Robert Braden, and David Borman. TCP Extensions for High Performance, May 1992. RFC 1323.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. {Kes88} Srinivasan Keshav. REAL: A Network Simulator. Technical Report 88/472, University of California Berkeley, 1988.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. {KL86} B. Kantor and P. Lapsley. Network News Transfer Protocol: A Proposed Standard for the Stream-Based Transmission of News, February 1986. RFC 977.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. {MA99} Matt Mathis and Mark Allman. Empirical Bulk Transfer Capacity, June 1999. Internet-Draft draft-ietf-ippm-btc-framework-01.txt.]]Google ScholarGoogle Scholar
  34. {Mah99} Jamshid Mahdavi. Enabling High Performance Data Transfers on Hosts. Technical report, Pittsburgh Supercomputer Center, June 1999. http://www.psc.edu/networking/perf_tune.html.]]Google ScholarGoogle Scholar
  35. {MD90} Jeffrey C. Mogul and Steve Deering. Path MTU Discovery, November 1990. RFC 1191.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. {MDM96} Jack McCann, Steve Deering, and Jeffrey C. Mogul. Path MTU Discovery for IP version 6, August 1996. RFC 1981.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. {MF95} Steven McCanne and Sally Floyd. NS (Network Simulator), 1995. URL http://www-nrg.ee.lbl.gov.]]Google ScholarGoogle Scholar
  38. {MM96a} Matt Mathis and Jamshid Mahdavi. Diagnosing Internet Congestion with a Transport Layer Performance Tool. In Proceedings of INET, June 1996.]]Google ScholarGoogle Scholar
  39. {MM96b} Matt Mathis and Jamshid Mahdavi. Forward Acknowledgment: Refining TCP Congestion Control. In ACM SIGCOMM, August -1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. {MM96c} Matt Mathis and Jamshid Mahdavi. TCP Rate-Halving with Bounding Parameters. Technical report, Pittsburgh Supercomputer Center, October 1996. URL: http://www.psc.edu/networking/papers/FACKnotes/current/.]]Google ScholarGoogle Scholar
  41. {MMFR96} Matt Mathis, Jamshid Mahdavi, Sally Floyd, and Allyn Romanow. TCP Selective Acknowledgement Options, October 1996. RFC 2018.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. {Mog92} Jeffrey C. Mogul. Observing TCP Dynamics in Real Networks. In ACM SIGCOMM, pages 305-317, 1992.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. {MSML99} Matt Mathis, Jeff Semke, Jamshid Mahdavi, and Kevin Lahey. The Rate-Halving Algorithm for TCP Congestion Control, August 1999. Internet-Draft draft-mathis-tcp-ratehalving-00.txt (work in progress).]]Google ScholarGoogle Scholar
  44. {MSMO97} Matt Mathis, Jeff Semke, Jamshid Mahdavi, and Teunis Ott. The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm. Computer Communication Review, 27(3), July 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. {MSMV99} Greg Minshall, Yasushi Saito, Jeffrey C. Mogul, and Ben Verghese. Application Performance Pitfalls and TCP's Nagle Algorithm. In Workshop on Internet Server Performance, May 1999.]]Google ScholarGoogle Scholar
  46. {Nag84} John Nagle. Congestion Control in IP/TCP Internetworks, January 1984. RFC 896.]]Google ScholarGoogle Scholar
  47. {PAD+99} Vern Paxson, Mark Allman, Scott Dawson, William Fenner, Jim Griner, Ian Heavens, Kevin Lahey, Jeff Semke, and Bernie Volz. Known TCP Implementation Problems, March 1999. RFC 2525.]]Google ScholarGoogle Scholar
  48. {Pax97a} Vern Paxson. Automated Packet Trace Analysis of TCP Implementations. In ACM SIGCOMM, September 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. {Pax97b} Vern Paxson. End-to-End Internet Packet Dynamics. In ACM SIGCOMM, September 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. {Pax97c} Vern Paxson. Measurements and Analysis of End-to-End Internet Dynamics. Ph.D. thesis, University of California Berkeley, 1997.]]Google ScholarGoogle Scholar
  51. {PBS+96} Venkata Padmanabhan, Hari Balakrishnan, Keith Sklower, Elan Amir, and Randy Katz. Networking Using Direct Broadcast Saellite. In Proceedings of the First International Workshop on Satellite-based Information Services (WOSBIS), November 1996.]]Google ScholarGoogle Scholar
  52. {PF95} Vern Paxson and Sally Floyd. Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, 3(3):226-244, June 1995.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. {PF97} Vern Paxson and Sally Floyd. Why We Don't Know How to Simulate the Internet. In Proceedings of the 1997 Winter Simulation Conference,December 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. {PMAM98} Vern Paxson, Jamshid Mahdavi, Andrew Adams, and Matt Mathis. An Architecture for Large-Scale Internet Measurement. IEEE Communications, 1998.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. {Pos81} Jon Postel. Transmission Control Protocol, September 1981. RFC 793.]]Google ScholarGoogle Scholar
  56. {PR85} Jon Postel and Joyce Reynolds. File Tranfer Protocol (FTP), October 1985. RFC 959.]]Google ScholarGoogle Scholar
  57. {PS98} Steve Parker and Chris Schmechel. Some Testing Tools for TCP Implementors, August 1998. RFC 2398.]]Google ScholarGoogle Scholar
  58. {RF99} K. K. Ramakrishnan and Sally Floyd. A Proposal to Add Explicit Congestion Notification (ECN) to IP, January 1999. RFC 2481.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. {Ric99} Alan Richard. ISINT Performance Validation Test Report. Technical Report CR-209407, NASA Glenn Research Center, June 1999.]]Google ScholarGoogle Scholar
  60. {Riz97} Luigi Rizzo. Dummynet: A Simple Approach to the Evaluation of Network Protocols. Computer Communications Review, 27(1):31-41, January 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. {SMM98} Jeff Semke, Jamshid Mahdavi, and Matt Mathis. Automatic TCP Buffer Tuning. In ACM SIGCOMM, September 1998.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. {Tec} Third Millennium Technologies. OpNet Modeler. http://www.mil3.com.]]Google ScholarGoogle Scholar
  63. {TMW97} Kevin Thompson, Gregory Miller, and Rick Wilder. Wide-Area Internet Traffic Patterns and Characteristics. IEEE Network, 11(6):10-23, November/December 1997.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. {VH97} Vikram Visweswaraiah and John Heidemann. Improving Restart of Idle TCP Connections. Technical Report 97-661, University of Southern California, August 1997.]]Google ScholarGoogle Scholar

Index Terms

  1. On the effective evaluation of TCP
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGCOMM Computer Communication Review
        ACM SIGCOMM Computer Communication Review  Volume 29, Issue 5
        October 1999
        73 pages
        ISSN:0146-4833
        DOI:10.1145/505696
        Issue’s Table of Contents

        Copyright © 1999 Authors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 5 October 1999

        Check for updates

        Qualifiers

        • article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader