An experimental evaluation of rate-adaptive video players over HTTP

Abstract

Adaptive (video) streaming over HTTP is gradually being adopted by content and network service providers, as it offers significant advantages in terms of both user-perceived quality and resource utilization. In this paper, we first focus on the rate-adaptation mechanisms of adaptive streaming and experimentally evaluate two major commercial players (Smooth Streaming and Netflix) and one open-source player (Adobe's OSMF). We first examine how the previous three players react to persistent and short-term changes in the underlying network available bandwidth. Do they quickly converge to the maximum sustainable bitrate? We identify major differences between the three players and significant inefficiencies in each of them. We then propose a new adaptation algorithm, referred to as AdapTech Streaming, which aims to address the problems with the previous three players. In the second part of the paper, we consider the following two questions. First, what happens when two adaptive video players compete for available bandwidth in the bottleneck link? Can they share that resource in a stable and fair manner? And second, how does adaptive streaming perform with live content? Is the player able to sustain a short playback delay, keeping the viewing experience “live”?

Introduction

Video has long been viewed as the “next killer application”.¹ Over the last 20 years, the various instances of packet video have been thought of as demanding applications that would never work satisfactorily over best-effort IP networks. That pessimistic view actually led to the creation of novel network architectures and QoS mechanisms, which were not deployed in a large-scale, though. Eventually, over the last 3–4 years video-based applications, and video streaming in particular, have become utterly popular generating more than half of the aggregate Internet traffic. Perhaps, surprisingly though, video streaming today runs over IP without any specialized support from the network. This has become possible through the gradual development of highly efficient video compression methods, the penetration of broadband access technologies, and the development of adaptive video players that can compensate for the unpredictability of the underlying network through sophisticated rate-adaptation, playback buffering, and error recovery and concealment methods.

Another conventional wisdom has been that video streaming would never work well over TCP, due to the throughput variations caused by TCP's congestion control and the potentially large retransmission delays. As a consequence, most of the earlier video streaming research has assumed that the underlying transport protocol is UDP (or RTP over UDP), which considerably simplifies the design and modeling of adaptive streaming applications. In practice, however, two points became clear in the last few years. First, TCP's congestion control mechanisms and reliability requirement do not necessarily hurt the performance of video streaming, especially if the video player is able to adapt to large throughput variations. Second, the use of TCP, and of HTTP over TCP in particular, greatly simplifies the traversal of firewalls and NATs.

The first wave of HTTP-based video streaming applications used the simple progressive download method, in which a TCP connection simply transfers the entire movie file as quickly as possible. The shortcomings of that approach are many, however. One major issue is that all clients receive the same encoding of the video, despite the large variations in the underlying available bandwidth both across different clients and across time for the same client. This has recently led to the development of a new wave of HTTP-based streaming applications that we refer to as adaptive streaming over HTTP (For a general overview of video streaming protocols and adaptive streaming, refer to [2]). Several players, such as Microsoft's Smooth Streaming, Adobe OSMF, as well as the players developed or used by Netflix, Move Networks and others, use this approach. In adaptive streaming, the server keeps multiple profiles of the same video, encoded in different bitrates and quality levels. Further, the video object is partitioned in chunks or fragments, typically a few seconds long. A player can then request different chunks at different encoding bitrates, depending on the underlying network conditions. Notice that it is the player that decides what bitrate to request for any chunk, improving server-side scalability. Another benefit of this approach is that the player can control its playback buffer size by dynamically adjusting the rate at which new chunks are requested.

Adaptive streaming over HTTP is a new technology. It is not yet clear whether the existing commercial players perform well, especially under dynamic network conditions. Further, the complex interactions between TCP's congestion control and the application's rate-adaptation mechanisms create a “nested double feedback loop”—the dynamics of such interacting control systems can be notoriously complex and hard to predict. As a first step toward understanding and improving such video streaming mechanisms, this paper experimentally evaluates two major commercial video players over HTTP (Microsoft's Smooth Streaming [17] and the player used by Netflix [15]) and one open-source player (Adobe's OSMF [16]).

In the first part of the paper (3 Microsoft smooth streaming, 4 Netflix player, 5 Adobe OSMF), we examine how the previous three players react to persistent and short-term variations in the underlying network available bandwidth. Do they quickly converge to the maximum sustainable bitrate? We identify major differences between the three players and significant inefficiencies in each of them. Then, in Section 6, we propose a new adaptation algorithm, referred to as AdapTech Streaming and implemented in Adobe's OSMF v1.5 player, which aims to address the issues with the previous three players.

In the second part of the paper (Sections 7 and 8), we consider the following two questions. First, what happens when two adaptive video players compete for available bandwidth in the bottleneck link? Can they share that resource in a stable and fair manner? And second, how does adaptive streaming perform with live content? Is the player able to sustain a short playback delay, keeping the viewing experience “live”?

In Section 2, we describe our experimental approach, the various tests we perform for each player, and the metrics we focus on. 3 Microsoft smooth streaming, 4 Netflix player, 5 Adobe OSMF focus on the Smooth Streaming, Netflix, and OSMF players, respectively. The proposed rate adaptation algorithm, AdaptTech Streaming, is described in Section 6. Section 7 focuses on the competition effects that take place when two adaptive players share the same bottleneck. Section 8 focuses on live video using the Smooth Streaming player. In Section 9, we review the related work on adaptive video streaming. We summarize what we learn for each player and conclude the paper in Section 10.