Is streaming TV worth the wait?

Critical research

Professor Ramesh K. Sitaraman is a faculty member in the Computer Science Department at the University of Massachusetts at Amherst, where he is part of the Theoretical Computer Science group. Sitaraman and associates recently studied nearly 6.7 million streaming video viewers across the world who watched nearly 23 million videos play about 216 million minutes of streaming video over a 10-day time period. To help the research, Akamai gave the team access to millions of anonymous reports containing information detailing how users watch videos and the quality they experienced. The report was the first large-scale study of its type that directly relates to the quality of experience (QoE) to viewer behavior.

With the Akamai-provided metrics, Sitaraman and his associates devised a novel technique that divided the 6.7 million viewers into two groups. One group experienced a high QoE; the other had poor QoE with start failures, delayed starts or repeated freezes. Members of the high and poor QoE groups were paired to match as closely as possible with other identifiable factors such as geographic location, content watched and connection types. Thus, the primary difference between those paired was QoE.

At the risk of oversimplification, the essence of what Sitaraman discovered was scientific evidence of something most broadcast engineers have suspected for some time: It verified that if a video clip doesn’t load and play in two seconds or less, viewers will start clicking on something else. Within 5 seconds, approximately 25 percent of viewers will have given up. After 10 seconds, about 50 percent of waiting viewers are gone.

The research indicated that viewers on high-speed fiber connections have the highest expectations and the least amount of start-time patience. For those with fiber connections, 5 seconds is way too long. Mobile device viewers are the most patient, at least for now.

In his research, Sitaraman made four assertions. His first assersion is: “An increase in (normalized) rebuffer delay can cause a decrease in play time.” Research indicated that play times and rebuffering times are inversely correlated and further, for even 1 percent of the video duration can cause viewers to watch 5 percent fewer minutes of the video.

His second assertion is: “Viewers are less tolerant of startup delay for short videos in comparison to longer videos. A viewer watching a short video such as a news clip is 11.5 percent more likely to abandon sooner during startup than its matching pair watching a long video, such as a movie.” Viewers are more patient if they perceive a higher value service is worth the wait. In streaming video, perceived value generally increases with the length of the video.

His third assertion is: “Viewers watching videos on a better connected computer or device have less patience for startup delay and so abandon sooner. For instance, a viewer with fiber broadband connectivity is 38.25 percent more likely to abandon sooner during startup than a similar viewer with mobile connectivity.” Viewers’ patience depends on their expectations.

His fourth and final assertion is: “A viewer who experienced a failed visit is 27 percent less likely to return to the content provider’s site to view more videos the next day than a similar viewer who did not experience a failed visit.” That’s not good news because it means that more than one-fourth visitors who had a QoE issue last time are less likely to give your website a second chance.

The author would like to thank Sitaraman for his advice and assistance in preparing this tutorial.

Professor Sitaraman’s research paper "Network Performance: Does It Really Matter To Users And By How Much?" is available here.

For a more in-depth treatment, see "Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs.”