OpenFARM plug-in framework
Nov 1, 2010 12:00 PM, By Filipe Martins and Jeremy Blythe
The software provides an open, efficient approach to content analysis.
Figure 1. The conceptual design of OpenFARM
Select figure to enlarge.
All broadcasters appreciate the freedom to expand their equipment infrastructure with hardware and software of their own choosing. Even the tidiest and most coordinated broadcast equipment bays house devices added to meet a requirement not anticipated at the time of original system installation.
So what happens when a television channel decides that it would like capabilities unforeseen during the original specification of its asset management system? Obviously the original designer is contacted and asked if a supplementary software or hardware is available to provide these new features. In many instances, the extra module will already have been designed to meet requests from other customers and can therefore be added at short notice.
In mid-2009, Pharos, a developer of broadcast content management systems, began development of the Open Framework for the Analysis of Rich Media (OpenFARM) project with Kingston University under the umbrella of Knowledge Transfer Partnership, a scheme partly funded by the British government to encourage collaboration between businesses and universities. The objective was not to design a product or toolset specific to Pharos, but one that would be open to any software designer, any manufacturer or any broadcaster that wanted to develop asset management plug-ins.
OpenFARM offers broadcasters a framework to add third-party software modules to their content management systems in essentially the same way that they select hardware for their central apparatus room. At its center is the construction of algorithms to identify, extract and interact with metadata in, for example, the video signal stream. Implementing the framework means that individuals or organizations wanting to develop such algorithms in the future can concentrate on their specific objective without having to build an entire operating framework.
The need
An uncataloged media asset database typically contains hundreds of hours of content that can be extremely costly to view, identify and catalog manually. Software developed using the framework can automate the cataloging process, saving thousands of euros. The framework makes it possible to extract all kinds of information (metadata) from video and audio media assets such as silent periods in an audio track, scene cuts, embedded text, logos and fades to black.
If a broadcast channel wanted to incorporate video analysis, such as a scene cut detector or a face recognition feature, into its asset management system but did not want to spend hundreds of hours in development and integration, the framework can provide easy integration in just a few hours.
Radical changes in the workflow of a media management system traditionally imply retesting and redesigning the software, or even major changes to the overall system architecture. The framework decouples all this by managing the video analysis components itself and feeding the content to the user/application only when queried, avoiding any proliferation of extra databases and third-party dependencies.
Key features
OpenFARM was used to implement a scene cut detector in Pharos Mediator.
The framework allows easy integration of media analysis components into broadcast system software. It is open source and published under a Massachusetts Institute of Technology license that allows it to be modified, extended and shipped with commercial applications.
The framework permits fast integration of advanced analysis components — such as scene cut detection, text extraction and speech detection — into any broadcast system software, regardless of the platform or programming language in which it was developed. Features such as a template pattern allow developers to design and develop analysis component plug-ins without being coupled to specific programming languages. Some of the frameworks features are:
It is free and open source, and it is written in Java and distributed across multiple machines.
It provides real-time media analysis.
It is database independent. All indexing metadata is returned to the client application (the broadcast software), or cached until such time that the client asks for it.
Concurrent analysis of media is coordinated by a single entry point (OpenFARM Manager) that is integrated with broadcast software.
It provides an internal soft GUI and a technology to generate client code for a specific broadcast application.
Multiple machines can be used to run the analysis components. Indexing tasks can be added to the schedule easily.
Load balancing is provided on the available processors for cost-effective and fast content analysis.
The analysis components can be written in practically any language (such as C, C++, C#, Java or Python) to run on any operating system, including Linux, and Windows.
Implementation
There are established methods for the identification of specific objects, categories, events and actions from video and audio data. The recognition of objects in still images, given a training image or set, has received much attention. In the context of content-based retrieval, the user may supply an image of the object with the intention of retrieving images of similar objects. The motion depicted in video data provides an additional dimension to use in content-based retrieval, enabling searches on actions and events.
There are many existing systems that embed content-based retrieval technology, working on either offline or online repositories. A key feature for commercial systems such as Picasa and Flickr is the detection and recognition of faces in photographic albums. MPEG-7 description schemes provide a standard means of representing metadata.
OpenFARM architecture comes in three stages: analysis, management and deployment. Each of these stages encompasses a set of units responsible for a specific type of processing. The analysis stage is responsible for loading the analysis components plug-ins and scaling them across a single or multiple platforms. The management stage is responsible for coordinating the analysis stage units and constitutes a single entry point between the user/application and the framework. The deployment stage is the key code to OpenFARM to be implemented in the media management software. Also part of this stage is a soft GUI that provides a control center to the framework. Table 1 indicates the types of analysis that are expected to be supported. These may be written by third-party developers and plugged in to the broadcast system using the framework designed to address the following requirement listed in the table.
Continue on next page
| Want to use this article? Click here for options! |  |
