org.openimaj.video.processing.shotdetector

Class VideoShotDetector<I extends Image<?,I>>

java.lang.Object

org.openimaj.video.Video<T>

org.openimaj.video.processor.VideoProcessor<I>

org.openimaj.video.processing.shotdetector.VideoShotDetector<I>

Type Parameters:

I - The type of image

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<I>, VideoDisplayListener<I>

Direct Known Subclasses:

CombiShotDetector, HistogramVideoShotDetector, LocalHistogramVideoShotDetector

public abstract class VideoShotDetector<I extends Image<?,I>>
extends VideoProcessor<I>
implements VideoDisplayListener<I>

Video shot detector class implemented as a video display listener. This means that shots can be detected as the video plays. The class also supports direct processing of a video file (with no display).

The default threshold boundary should be set by implementing methods as the distances returned by those implementations will only sensibly understand where the threshold should be.

Only the last keyframe is stored during processing, so if you want to store a list of keyframes you must store this list yourself by listening to the ShotDetected event which provides a VideoKeyframe which has a timecode and an image. Each event will receive the same VideoKeyframe instance containing different information. Use VideoKeyframe#clone() to make a copy.

Author:

David Dupplaw (dpd@ecs.soton.ac.uk)

Created:

1 Jun 2011

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	lastFrameWasBoundary Whether the last processed frame was a boundary
protected double	threshold The threshold to use to determine a shot boundary - this default is arbitrary

Constructor Summary

Constructors

Constructor and Description
VideoShotDetector() This constructor assumes that you will set the number of frames per second your video uses (using setFPS(double)) when you know what that will be, otherwise your timecodes will all be messed up.
VideoShotDetector(double fps) Default constructor that allows the processor to be used ad-hoc on frames from any source.
VideoShotDetector(Video<I> video) Constructor that takes the video file to process.
VideoShotDetector(Video<I> video, boolean display) Default constructor that takes the video file to process and whether or not to display the video as it's being processed.

Method Summary

Modifier and Type	Method and Description
void	addShotDetectedListener(ShotDetectedListener<I> sdl) Add the given shot detected listener to the list of listeners in this object
void	afterUpdate(VideoDisplay<I> display) Called when the video display is updated and the next frame is displayed
void	beforeUpdate(I frame) Called when the video is about to update
protected void	fireDifferentialCalculated(VideoTimecode vt, double d, I frame) Fired each time a differential is calculated between frames.
protected void	fireShotDetected(ShotBoundary<I> sb, VideoKeyframe<I> vk) Fire the event to the listeners that a new shot has been detected.
DoubleFV	getDifferentials() Get the differentials between frames (if storeAllDiff is true).
protected abstract double	getInterframeDistance(I thisFrame) Returns the inter-frame distance between this frame and the last.
VideoKeyframe<I>	getLastKeyframe() Returns the last video keyframe that was generated.
ShotBoundary<I>	getLastShotBoundary() Return the last shot boundary in the list.
List<ShotBoundary<I>>	getShotBoundaries() Get the list of shot boundaries that have been extracted so far.
double	getThreshold() Returns the current threshold value.
void	process() Process the video.
I	processFrame(I frame) Process a frame in this video.
void	removeShotDetectedListener(ShotDetectedListener<I> sdl) Remove the given shot detected listener from this object.
void	reset() No implementation.
void	setFindKeyframes(boolean k) Set whether to store keyframes of boundaries when they have been found.
void	setFPS(double fps) Set the frames per second value for the video being processed.
void	setStoreAllDifferentials(boolean d) Set whether to store differentials during the processing stage.
void	setThreshold(double threshold) Set the threshold that will determine a shot boundary.
boolean	wasLastFrameBoundary() Returns whether the last processed frame was a shot boundary - that is the last processed frame marks a new scene.

Methods inherited from class org.openimaj.video.processor.VideoProcessor

countFrames, getCurrentFrame, getFPS, getHeight, getNextFrame, getTimeStamp, getWidth, hasNextFrame, process, processingComplete

Methods inherited from class org.openimaj.video.Video

close, getCurrentFrameIndex, iterator, seek, setCurrentFrameIndex

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

lastFrameWasBoundary

protected boolean lastFrameWasBoundary

Whether the last processed frame was a boundary

threshold

protected double threshold

The threshold to use to determine a shot boundary - this default is arbitrary

Constructor Detail

VideoShotDetector

public VideoShotDetector()

This constructor assumes that you will set the number of frames per second your video uses (using setFPS(double)) when you know what that will be, otherwise your timecodes will all be messed up.

VideoShotDetector

public VideoShotDetector(double fps)

Default constructor that allows the processor to be used ad-hoc on frames from any source. The number of FPS is required so that timecodes can be generated for the shot boundaries. Be aware that if your source material does not have a specific number of frames per second then the timecodes will not have any meaning in the detected shot boundaries.

Parameters:

fps - The number of frames per second of the source material

VideoShotDetector

public VideoShotDetector(Video<I> video)

Constructor that takes the video file to process.

Parameters:

video - The video to process.

VideoShotDetector

public VideoShotDetector(Video<I> video,
                         boolean display)

Default constructor that takes the video file to process and whether or not to display the video as it's being processed.

Parameters:

video - The video to process

display - Whether to display the video during processing.

Method Detail

wasLastFrameBoundary

public boolean wasLastFrameBoundary()

Returns whether the last processed frame was a shot boundary - that is the last processed frame marks a new scene.

Returns:

Whether the last frame was a boundary.

process

public void process()

Process the video.

Overrides:

process in class VideoProcessor<I extends Image<?,I>>

afterUpdate

public void afterUpdate(VideoDisplay<I> display)

Called when the video display is updated and the next frame is displayed

Specified by:

afterUpdate in interface VideoDisplayListener<I extends Image<?,I>>

Parameters:

display - the video display

See Also:

VideoDisplayListener.afterUpdate(org.openimaj.video.VideoDisplay)

beforeUpdate

public void beforeUpdate(I frame)

Called when the video is about to update

Specified by:

beforeUpdate in interface VideoDisplayListener<I extends Image<?,I>>

Parameters:

frame - the frame about to be displayed

See Also:

VideoDisplayListener.beforeUpdate(org.openimaj.image.Image)

addShotDetectedListener

public void addShotDetectedListener(ShotDetectedListener<I> sdl)

Add the given shot detected listener to the list of listeners in this object

Parameters:

sdl - The shot detected listener to add

removeShotDetectedListener

public void removeShotDetectedListener(ShotDetectedListener<I> sdl)

Remove the given shot detected listener from this object.

Parameters:

sdl - The shot detected listener to remove

getLastShotBoundary

public ShotBoundary<I> getLastShotBoundary()

Return the last shot boundary in the list.

Returns:

The last shot boundary in the list.

getLastKeyframe

public VideoKeyframe<I> getLastKeyframe()

Returns the last video keyframe that was generated.

Returns:

The last video keyframe that was generated.

getInterframeDistance

protected abstract double getInterframeDistance(I thisFrame)

Returns the inter-frame distance between this frame and the last.

Returns:

The inter-frame distance

getShotBoundaries

public List<ShotBoundary<I>> getShotBoundaries()

Get the list of shot boundaries that have been extracted so far.

Returns:

The list of shot boundaries.

setThreshold

public void setThreshold(double threshold)

Set the threshold that will determine a shot boundary.

Parameters:

threshold - The new threshold.

getThreshold

public double getThreshold()

Returns the current threshold value.

Returns:

The current threshold

setFindKeyframes

public void setFindKeyframes(boolean k)

Set whether to store keyframes of boundaries when they have been found.

Parameters:

k - TRUE to store keyframes; FALSE otherwise

setStoreAllDifferentials

public void setStoreAllDifferentials(boolean d)

Set whether to store differentials during the processing stage.

Parameters:

d - TRUE to store all differentials; FALSE otherwise

getDifferentials

public DoubleFV getDifferentials()

Get the differentials between frames (if storeAllDiff is true).

Returns:

The differentials between frames as a List of Double.

processFrame

public I processFrame(I frame)

Process a frame in this video. The processor must determine itself what is to be done with the frame that is processed. It is suggest that subclass processors add listeners for processed frames if they are required. The implementation must process the frame in-place and the frame should be returned.

Specified by:

processFrame in class VideoProcessor<I extends Image<?,I>>

Parameters:

frame - The frame to process.

Returns:

the processed frame

See Also:

VideoProcessor.processFrame(org.openimaj.image.Image)

fireShotDetected

protected void fireShotDetected(ShotBoundary<I> sb,
                                VideoKeyframe<I> vk)

Fire the event to the listeners that a new shot has been detected.

Parameters:

sb - The shot boundary defintion

vk - The video keyframe

fireDifferentialCalculated

protected void fireDifferentialCalculated(VideoTimecode vt,
                                          double d,
                                          I frame)

Fired each time a differential is calculated between frames.

Parameters:

vt - The timecode of the differential

d - The differential value

frame - The different frame

reset

public void reset()

No implementation.

Overrides:

reset in class VideoProcessor<I extends Image<?,I>>

See Also:

VideoProcessor.reset()

setFPS

public void setFPS(double fps)

Set the frames per second value for the video being processed.

Parameters:

fps - The number of frames per second.