Real-time detection of commercials on television
For my undergraduate honours thesis I conducted research into the unbuffered real-time detection of commercials on television with a view to muting the volume when ads are being broadcast. The research itself dealt with examining the features required to enable robust real-time detection. I developed a sophisticated video analysis and processing framework to underpin experimentation and compilation of results.
The following screenshots show the system running live (click on one to see the full-res image):
TehDetector
I wrote a Windows-based video analysis and processing framework to underpin the research I undertook for my undergraduate thesis.
Some of the features it boasts:
- DirectShow-based: Enables analysis and process of any video/audio format that DShow supports.
- Seeking with frame-accuracy in an MPEG-2 stream: This handy feature I implemented, when used with certain filters, enables frame-accurate seeking (which otherwise is not possible with the standard filter graph (i.e. source filter and demultiplexer).
- Input live streaming TV: I use a Terrestrial Digital Video Broadcasting card to feed Teh Detector live video, which it processes in real-time. It (obviously) also supports offline analysis of stored content.
- Extensible detector architecture: Detectors (analysis components) can be written in C++ and added at will to the framework to extend its capabilities. I implemented several detectors in a hierarchy fashion to find television commercials in a live stream.
- Automatic frame caching: The framework automatically caches video frames and optimises analysis during runtime by also caching various calculations performed on an incoming frame.
- Event store/viewer: Detectors can output 'events' that describe a detected feature in the video, which is automatically stored and managed by the framework. These events can then later be reviewed in an intuitive manner.
- Lots of others: detector timing profiler, signal strength meter, playback rate control, filter graph management, bookmarking, frame dumping, extensive keyboard shortcuts...
Click on one of the following screenshots to see the full-res image:
The left-hand screen shows the main controller interface (middle) with the video preview above it. On the left side of the video preview is an example of a detector: the hard-cut detector. It outputs instantaneous information to the luminosity histogram on the right of the video preview,
and time-series information to the graph below the controller window. The right-hand screen shows the event viewer displaying four different detector (analysis) runs performed on portions of the input video. Here, each shows the before & after frame for a detected hard cut.
The controller window, video preview and event viewer can also be seen here. In addition, the DirectShow filter graph information window is visible below the controller, and on its right is the manual event input window. Overlaid upon the event viewer is the event list window, which displays events for each detector (analysis) run and enables a user to select the events/runs that should be shown, edited, etc.
