public class VideoGroundTruth extends Object
VideoGroundTruth.IdentifierProducer
to tag objects
within the scene. Each Identifiable
that the producer produces has a
set of states (annotations) which are given by the VideoGroundTruth.StateProvider
. The
tool outputs a list of annotations each linked to a list of frames (
VideoGroundTruth.IdentifiableVideoFrame
s) each of which contains a list of
Identifiable
s whose state changed on that frame. For example,
00:00:01:23 Scene 1 - Face 1 SPEAKING
00:00:04:12 Scene 1 - Face 1 NOT_SPEAKING
00:00:06:05 Scene 1 - Face 2 SPEAKING
00:00:10:01 Scene 1 - Face 2 NOT_SPEAKING
00:00:11:15 Scene 2 - Face 1 SPEAKING
00:00:14:00 Scene 2 - Face 1 NOT_SPEAKING
In this example there are two faces identified (within the first scene). Face
1 speaks between 00:00:01:23 (1 second 23 frames) and 00:00:04:12. Face 2
speaks between 00:00:06:05 and 00:00:10:01. The first face in another scene
begins speaking again at 00:00:11:15 until 00:00:14:00. Note that Face 1 in
Scenes 1 and 2 may be different people (they may not too). The identifier
provider does all it can in the situation to provide unique identifiers. The
meaning of Face 1 and 2 depends on the implementation of the identifier
provider. SPEAKING and NOT_SPEAKING is given by the VideoGroundTruth.StateProvider
and
the timecodes are given by the VideoGroundTruth.IdentifiableVideoFrame
(created by
default).
Annotations can also include end times. So the above could be represented as:
00:00:01:23-00:00:04:11 Scene 1 - Face 1 SPEAKING
00:00:04:12-00:00:06:04 Scene 1 - Face 1 NOT_SPEAKING
etc.
The user interface provided is based on a VideoPlayer
. The
VideoGroundTruth.StateProvider
can provide its own user interface for selecting the
current state, or it can provide a key listener which will be automatically
added to the video player controls. The user should be in the state provider
loop.
The object for annotating is selected by the user of this class. Also, the
means for updating the state of any selected identifiable is also determined
by the user of this class. When the state of an identifiable changes, the
updateIdentifiable(Identifiable)
method should be called, which will
query the state provider for the current state of the identifiable. This will
then be added to the dataset for the current video timecode.
The class provides a writeDataset(File)
method for writing the
generated dataset to a text file. The file is formatted by the
VideoGroundTruth.AnnotatedIdentifiable.toString()
method in the form:
id@start>end:[comma-separated-tags]
This implies that the identifier cannot include @ symbols and that tags may
not contain commas. The timecodes are HrsMinSecFrameTimecode
objects
represented as strings (Hrs:Mins:Secs:Frames).Modifier and Type | Class and Description |
---|---|
static class |
VideoGroundTruth.AnnotatedIdentifiable
An identifiable that has been annotated with states (tags) between
specific times.
|
static class |
VideoGroundTruth.IdentifiableVideoFrame
An identifiable video frame that uses the timecode of the frame to
identify the frame.
|
static interface |
VideoGroundTruth.IdentifierProducer
An interface for objects which produce identifiers to stamp.
|
static interface |
VideoGroundTruth.StateProvider
Provides states for identifiables.
|
Constructor and Description |
---|
VideoGroundTruth(Video<MBFImage> video,
AudioStream audio,
VideoGroundTruth.IdentifierProducer idProd,
VideoGroundTruth.StateProvider sp)
Constructor
|
VideoGroundTruth(Video<MBFImage> video,
AudioStream audio,
VideoGroundTruth.StateProvider sp)
Constructor that provides an identifier producer that returns the current
frame of the video; that is the state provider tags frames of video
rather than objects within frames of video.
|
VideoGroundTruth(Video<MBFImage> video,
VideoGroundTruth.StateProvider sp)
Constructor that provides an identifier producer that returns the current
frame of the video.
|
Modifier and Type | Method and Description |
---|---|
void |
addToDataset(VideoGroundTruth.AnnotatedIdentifiable ai)
Add the given annotated identifiable to the dataset
|
VideoPlayer<MBFImage> |
getVideoPlayer()
Returns the video player component being used to play the video.
|
void |
loadDataset(File file)
Loads a dataset created using
writeDataset(File) into this
class's dataset member so that annotation can be continued. |
void |
run()
Start the process.
|
void |
updateIdentifiable(Identifiable i)
Force the given identifiable to be updated in the dataset for the current
time.
|
void |
updateIdentifiableRegion(Identifiable i,
VideoTimecode start,
VideoTimecode end)
Add an identifiable time region that will be annotated with the current
state.
|
void |
writeDataset(File file)
Writes the created dataset to the given file
|
public VideoGroundTruth(Video<MBFImage> video, VideoGroundTruth.StateProvider sp)
video
- The video to ground truthsp
- The state providerpublic VideoGroundTruth(Video<MBFImage> video, AudioStream audio, VideoGroundTruth.StateProvider sp)
video
- The video to ground truthaudio
- The audio to playsp
- The state providerpublic VideoGroundTruth(Video<MBFImage> video, AudioStream audio, VideoGroundTruth.IdentifierProducer idProd, VideoGroundTruth.StateProvider sp)
video
- The video to ground truthaudio
- The audio to playidProd
- The identifier producersp
- The state providerpublic void run()
public VideoPlayer<MBFImage> getVideoPlayer()
public void updateIdentifiable(Identifiable i)
i
- The identifiablepublic void updateIdentifiableRegion(Identifiable i, VideoTimecode start, VideoTimecode end)
i
- The identifiablestart
- The start timestampend
- The end timestamppublic void addToDataset(VideoGroundTruth.AnnotatedIdentifiable ai)
ai
- The annotated identifiablepublic void writeDataset(File file) throws IOException
file
- The file to write the dataset to.IOException
- If the file could not be writtenpublic void loadDataset(File file) throws IOException
writeDataset(File)
into this
class's dataset member so that annotation can be continued. Note that the
identifiables that are created when reading back in are anonymous
identifiable classes and are not necessarily of the same type as the
identifiable which was saved. It will only include the identifier of the
object retrievable with Identifiable.getID()
.file
- The file to read fromIOException
- If the file cannot be read