The Winter Consumer Electronics Show (CES) is in our collective rear view mirror. In terms of TV and video, the show is no longer about hardware; it’s about software. While studying reports on this winter’s CES, I was reminded of my first trip to a Winter CES in the mid '70s. RCA had an exhibit booth and a big meeting room inside the Las Vegas Convention Center filled with the latest line-up of RCA XL-100 TV sets. I asked a salesperson a question about picture quality. If you keep in mind that this was the '70s, I’ll paraphrase his answer: "The lady of the house usually chooses the TV, and her brand and model choice is usually based on size and cabinet style. Picture quality is never an issue."
Fast forward to last month’s CES. RCA is barely a footnote. So are the XL100s, VHS machines and TV consoles that ended up in basements, garages and landfills. Today’s TV and video recorders/players are commodity items. If you want your TV in a Mediterranean cabinet, you’ll have to buy or build one yourself. Picture quality is defined by pixels, bits and industry standards, not by design, cost-cutting and analog technology as it once was. Other than size, most new TVs look nearly the same, power on or power off. That’s my opinion, anyway. Now, let’s talk about something a little closer to home.
Streaming or RF?
Broadcasting traditional over the air (OTA) RF is like spraying a fire hose in the dark. You’ll probably soak some viewers, hit some others with mist and maybe even get an estimate of how many, but you will not likely find out who, where or how wet they got. Questions of exactly who is in the audience have perplexed sales managers, station managers, advertisers and owners since before Arthur Nielsen began providing statistical analysis to radio broadcasters about 80 years ago. In 1950, Nielsen added TV ratings, using essentially the same methods Nielsen and his organization had developed for radio. The question of who is watching remains one of the largest unanswered questions in the broadcasting industry.
Besides broadcasting over the air, most DTV stations are maintaining sophisticated websites with links to streaming video clips. Some are hosted locally; most are in the cloud. Besides the obvious, the main difference between OTA broadcasting and streaming is that you can harvest all kinds of detailed metrics and statistics from streaming viewers to help shed light on the “who, where and how long” questions.
The other difference is that streaming has unique ongoing costs most local broadcasters are still learning about. While broadcasters seems to agree that a strong Internet presence is necessary, fewer at stations and groups seem to agree on how much to budget for it or how good it should be. If good enough is good enough, how do you define "good enough"?
The introduction of HTML5, Flash, Silverlight and Quicktime/iOS made it possible for an integrated analytics plug-in to run inside media players. Such a plug-in can report user actions such as start, stop, rewind and abandonment, while measuring performance metrics such as bandwidth, startup delays and rebuffering events.
Analytics technology can be highly useful in building a case that good enough is not good enough. If, for example, a content provider can use the plug-in to scientifically determine that video freezes cause people to view 5 percent fewer minutes of content, that fact can be correlated into lost ad impressions and, ultimately, lost revenues. When scientific data is converted to lost dollars and cents, it can easily establish a solid foundation to justify investing in better performing network services.
Akamai Technologies is a popular Internet content delivery network that also makes a widely used media analytics plug-in. According to Akamai’s website, its global traffic is about 21 million HTTP hits per second. Not coincidentally, Akamai is also based in Cambridge, MA, home of MIT.
One of Akamai’s co-founders, Tom Leighton, was an applied mathematics professor and head of the Algorithms Group at MIT’s Laboratory for Computer Science. The other co-founder, Daniel Lewin, was an MIT graduate student. Together, they developed algorithms to control dynamic routing of content and founded Akamai.
Paul Sagan, former president of Time Inc. New Media, joined Akamai in 1999. Sagan founded Road Runner and also helped start NY1 News. He was named Akamai’s COO and eventually became president. That same year, Akamai began commercial service. About two years later, co-founder Daniel Lewin died on Sept. 11, 2001. He was aboard American Airlines flight 11.
Professor Ramesh K. Sitaraman is a faculty member in the Computer Science Department at the University of Massachusetts at Amherst, where he is part of the Theoretical Computer Science group. Sitaraman and associates recently studied nearly 6.7 million streaming video viewers across the world who watched nearly 23 million videos play about 216 million minutes of streaming video over a 10-day time period. To help the research, Akamai gave the team access to millions of anonymous reports containing information detailing how users watch videos and the quality they experienced. The report was the first large-scale study of its type that directly relates to the quality of experience (QoE) to viewer behavior.
With the Akamai-provided metrics, Sitaraman and his associates devised a novel technique that divided the 6.7 million viewers into two groups. One group experienced a high QoE; the other had poor QoE with start failures, delayed starts or repeated freezes. Members of the high and poor QoE groups were paired to match as closely as possible with other identifiable factors such as geographic location, content watched and connection types. Thus, the primary difference between those paired was QoE.
At the risk of oversimplification, the essence of what Sitaraman discovered was scientific evidence of something most broadcast engineers have suspected for some time: It verified that if a video clip doesn’t load and play in two seconds or less, viewers will start clicking on something else. Within 5 seconds, approximately 25 percent of viewers will have given up. After 10 seconds, about 50 percent of waiting viewers are gone.
The research indicated that viewers on high-speed fiber connections have the highest expectations and the least amount of start-time patience. For those with fiber connections, 5 seconds is way too long. Mobile device viewers are the most patient, at least for now.
In his research, Sitaraman made four assertions. His first assersion is: “An increase in (normalized) rebuffer delay can cause a decrease in play time.” Research indicated that play times and rebuffering times are inversely correlated and further, for even 1 percent of the video duration can cause viewers to watch 5 percent fewer minutes of the video.
His second assertion is: “Viewers are less tolerant of startup delay for short videos in comparison to longer videos. A viewer watching a short video such as a news clip is 11.5 percent more likely to abandon sooner during startup than its matching pair watching a long video, such as a movie.” Viewers are more patient if they perceive a higher value service is worth the wait. In streaming video, perceived value generally increases with the length of the video.
His third assertion is: “Viewers watching videos on a better connected computer or device have less patience for startup delay and so abandon sooner. For instance, a viewer with fiber broadband connectivity is 38.25 percent more likely to abandon sooner during startup than a similar viewer with mobile connectivity.” Viewers’ patience depends on their expectations.
His fourth and final assertion is: “A viewer who experienced a failed visit is 27 percent less likely to return to the content provider’s site to view more videos the next day than a similar viewer who did not experience a failed visit.” That’s not good news because it means that more than one-fourth visitors who had a QoE issue last time are less likely to give your website a second chance.
The author would like to thank Sitaraman for his advice and assistance in preparing this tutorial.
Professor Sitaraman’s research paper "Network Performance: Does It Really Matter To Users And By How Much?" is available here.
For a more in-depth treatment, see "Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs.”