Chen Tian
ABSTRACT
As live streaming networks grow in scale and complexity, they are becoming increasingly difficult to evaluate. Existing evaluation methods including lab/testbed testing, simulation, and theoretical modeling, lack either scale or realism. The industrial practice of gradually-rolling-out in a testing channel is lacking in controllability and protection when experimental algorithms fail, due to its passive approach. In this paper, we design a novel system called ShadowStream that introduces evaluation as a built-in capability in production Internet live streaming networks. ShadowStream introduces a simple, novel, transparent embedding of experimental live streaming algorithms to achieve safe evaluations of the algorithms during large-scale, real production live streaming, despite the possibility of large performance failures of the tested algorithms. ShadowStream also introduces transparent, scalable, distributed experiment orchestration to resolve the mismatch between desired viewer behaviors and actual production viewer behaviors, achieving experimental scenario controllability. We implement ShadowStream based on a major Internet live streaming network, build additional evaluation tools such as deterministic replay, and demonstrate the benefits of ShadowStream through extensive evaluations.
Supplemental Material
- Cisco Visual Networking Index: Forecast and Methodology, 2011--2016. www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/%white_paper_c11--481360.pdf.Google Scholar
- PowerBoost. broadbandreports.com/shownews/75298.Google Scholar
- S. Agarwal, J. P. Singh, A. Mavlankar, P. Baccichet, and B. Girod. Performance and Quality-of-Service Analysis of a Live P2P Video Multicast Session on the Internet. In IWQoS 2008.Google Scholar
- G. Altekar and I. Stoica. ODR: Output-Deterministic Replay for Multicore Debugging. In ACM SOSP 2009. Google ScholarDigital Library
- IETF ALTO. datatracker.ietf.org/wg/alto/charter/.Google Scholar
- S. Banerjee, B. Bhattacharjee, and C. Kommareddy. Scalable Application Layer Multicast. In SIGCOMM 2002. Google ScholarDigital Library
- A. Bavier, N. Feamster, M. Huang, L. Peterson, and J. Rexford. In VINI Veritas: Realistic and Controlled Network Experimentation. In SIGCOMM 2006. Google ScholarDigital Library
- T. Bonald, L. Massoulie, F. Mathieu, D. Perino, and A. Twigg. Epidemic Live Streaming: Optimal Performance Trade-offs. In SIGMETRICES 2008. Google ScholarDigital Library
- Broadcast Delay. en.wikipedia.org/wiki/Broadcast_delay.Google Scholar
- M. Castro, P. Druschel, A. marie Kermarrec, A. Nandi, A. Rowstron, and A. Singh. SplitStream: High-bandwidth Multicast in Cooperative Environments. In ACM SOSP 2003. Google ScholarDigital Library
- H. Chang, S. Jamin, and W. Wang. Live Streaming Performance of the Zattoo Network. In IMC 2009. Google ScholarDigital Library
- B. Chun, D. Culler, T. Roscoe, A. Bavier, L. Peterson, M. Wawrzoniak, and M. Bowman. PlanetLab: an Overlay Testbed for Broad-coverage Services. ACM SIGCOMM CCR, 33(3):3--12, 2003. Google ScholarDigital Library
- D. R. Cox and P. A. W. Lewis. The Statistical Analysis of Series of Events. Methuen, 1966.Google ScholarCross Ref
- M. Dischinger, K. P. Gummadi, A. Haeberlen, S. Saroiu, and I. Beschastnikh. SatelliteLab: Adding Heterogeneity to Planetary-Scale Network Testbeds. In SIGCOMM 2008. Google ScholarDigital Library
- M. Dischinger, A. Haeberlen, K. P. Gummadi, and S. Saroiu. Characterizing Residential Broadband Networks. In IMC 2007. Google ScholarDigital Library
- D. Geels, G. Altekar, P. Maniatis, T. Roscoe, and I. Stoica. Friday: Global Comprehension for Distributed Replay. In NSDI 2007. Google ScholarDigital Library
- GNU Radio. www.gnu.org/software/gnuradio/.Google Scholar
- D. Gupta, K. V. Vishwanath, and A. Vahdat. DieCast: Testing Distributed Systems with an Accurate Scale Model. In NSDI 2008. Google ScholarDigital Library
- Y. Huang, T. Z. J. Fu, D.-M. Chiu, J. C. S. Lui, and C. Huang. Challenges, Design and Analysis of a Large-scale P2P-VoD System. In SIGCOMM 2008. Google ScholarDigital Library
- M. S. Kim, T. Kim, Y.-J. Shin, S. S. Lam, and E. J. Powers. A Wavelet-Based Approach to Detect Shared Congestion. In SIGCOMM 2004. Google ScholarDigital Library
- E. Kohler, R. Morris, B. Chen, J. Jannotti, and M. F. Kaashoek. The Click Modular Router. ACM Trans. on Computer Systems, 18(3):263--297, Aug. 2000. Google ScholarDigital Library
- D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat. Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh. In ACM SOSP 2003. Google ScholarDigital Library
- C. Kreibich, N. Weaver, B. Nechaev, and V. Paxson. Netalyzr: Illuminating the Edge Network. In IMC 2010. Google ScholarDigital Library
- R. Krishnan, H. V. Madhyastha, S. Jain, S. Srinivasan, A. Krishnamurthy, T. Anderson, and J. Gao. Moving Beyond End-to-End Path Information to Optimize CDN Performance. In IMC 2009. Google ScholarDigital Library
- R. Kumar, Y. Liu, and K. Ross. Stochastic Fluid Theory for P2P Streaming Systems. In INFOCOM 2007.Google ScholarDigital Library
- L. Leonini, E. Riviere, and P. Felber. SPLAY: Distributed Systems Evaluation Made Simple. In NSDI 2009. Google ScholarDigital Library
- B. Li, G. Y. Keung, C. Lin, J. Liu, and X. Zhang. Inside the New Coolstreaming: Principles, Measurements and Performance Implications. In INFOCOM 2008.Google Scholar
- Z. Liu, C. Wu, B. Li, and S. Zhao. Why Are Peers Less Stable in Unpopular P2P Streaming Channels? In Networking 2009. Google ScholarDigital Library
- N. Magharei and R. Rejaie. PRIME: Peer-to-Peer Receiver-drIven MEsh-based Streaming. In INFOCOM 2007.Google Scholar
- A. Mansy and M. Ammar. Analysis of Adaptive Streaming for Hybrid CDN/P2P Live Video Systems. In ICNP 2011. Google ScholarDigital Library
- V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, A. E. Mohr, and E. E. Mohr. Chainsaw: Eliminating Trees from Overlay Multicast. In IPTPS 2005. Google ScholarDigital Library
- F. Picconi and L. Massoulie. Is there a future for mesh-based live video streaming? In P2P 2008. Google ScholarDigital Library
- PPLive User Comments. tieba.baidu.com/f?kz=700224794.Google Scholar
- S. Shalunov, G. Hazel, J. Iyengar, and M. Kuehlewind. Low Extra Delay Background Transport (LEDBAT). IETF Draft.Google Scholar
- R. Sherwood, G. Gibb, K. kiong Yap, M. Casado, N. Mckeown, and G. Parulkar. Can the Production Network be the Testbed. In OSDI 2010. Google ScholarDigital Library
- S. Sundaresan, W. de Donato, N. Feamster, R. Teixeira, S. Crawford, and A. Pescape. Broadband Internet Performance: A View From the Gateway. In SIGCOMM 2011. Google ScholarDigital Library
- C. Tuttle, A. C. Snoeren, and A. Vahdat. PlanetLab Application Management Using Plush. Operating Systems Review, 40(1), Nov. 2006. Google ScholarDigital Library
- I. Ullah, G. Bonnet, G. Doyen, and D. Gaïti. Modeling User Behavior in P2P Live Video Streaming Systems through a Bayesian Network. In MAMNS 2010. Google ScholarDigital Library
- V. Venkataraman, P. Francis, and J. Calandrino. Chunkyspread: Multi-tree Unstructured Peer-to-Peer Multicast. In IPTPS 2006.Google Scholar
- M. Welsh, D. Culler, and E. Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. In ACM SOSP 2001. Google ScholarDigital Library
- B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An Integrated Experimental Environment for Distributed Systems and Networks. In OSDI 2002. Google ScholarDigital Library
- C. Wu, B. Li, and S. Zhao. Diagnosing Network-wide P2P Live Streaming Inefficiencies. In INFOCOM 2009.Google ScholarCross Ref
- C. Wu, B. Li, and S. Zhao. Multi-channel Live P2P Streaming: Refocusing on Servers. In INFOCOM 2008.Google ScholarCross Ref
- H. Xie, Y. R. Yang, A. Krishnamurthy, Y. Liu, and A. Silberschatz. P4P: Provider Portal for Applications. In SIGCOMM 2008. Google ScholarDigital Library
- H. Yin, X. Liu, T. Zhan, V. Sekar, F. Qiu, C. Lin, H. Zhang, and B. Li. Design and Deployment of a Hybrid CDN-P2P System for Live Video Streaming: Experiences with LiveSky. In ACM MultiMedia 2009. Google ScholarDigital Library
- Y. Zhou, D.-M. Chiu, and J. C. Lui. A Simple Model for Analyzing P2P Streaming Protocols. In ICNP 2007.Google ScholarCross Ref
Index Terms
- ShadowStream: performance evaluation as a capability in production internet live streaming networks
Recommendations
ShadowStream: performance evaluation as a capability in production internet live streaming networks
Special october issue SIGCOMM '12As live streaming networks grow in scale and complexity, they are becoming increasingly difficult to evaluate. Existing evaluation methods including lab/testbed testing, simulation, and theoretical modeling, lack either scale or realism. The industrial ...
High resolution live streaming with the HYDRA architecture
First anniversary issueDigital continuous media (CM) are now well established as an integral part of many applications. With highdefinition (HD) displays becoming increasingly common and large network bandwidth available, high-quality video streaming has become feasible, and ...
Data Streams with Bounded Deletions
PODS '18: Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database SystemsTwo prevalent models in the data stream literature are the insertion-only and turnstile models. Unfortunately, many important streaming problems require a Θ(log(n)) multiplicative factor more space for turnstile streams than for insertion-only streams. ...
Comments