org.openimaj.image.feature.local.aggregate

Class FisherVector<T>

java.lang.Object

org.openimaj.image.feature.local.aggregate.FisherVector<T>

Type Parameters:

T - Primitive array type of the ArrayFeatureVectors used by the LocalFeatures that will be processed.

All Implemented Interfaces:

VectorAggregator<ArrayFeatureVector<T>,FloatFV>

@Reference(type=Inproceedings,author={"Perronnin, F.","Dance, C."},title="Fisher Kernels on Visual Vocabularies for Image Categorization",year="2007",booktitle="Computer Vision and Pattern Recognition, 2007. CVPR \'07. IEEE Conference on",pages={"1","8"},customData={"keywords","Gaussian processes;gradient methods;image classification;Fisher kernels;Gaussian mixture model;generative probability model;gradient vector;image categorization;pattern classification;visual vocabularies;Character generation;Feeds;Image databases;Kernel;Pattern classification;Power generation;Signal generators;Spatial databases;Visual databases;Vocabulary","doi","10.1109/CVPR.2007.383266","ISSN","1063-6919"}) @Reference(type=Inproceedings,author={"Perronnin, Florent","S\'{a}nchez, Jorge","Mensink, Thomas"},title="Improving the Fisher Kernel for Large-scale Image Classification",year="2010",booktitle="Proceedings of the 11th European Conference on Computer Vision: Part IV",pages={"143","","156"},url="http://dl.acm.org/citation.cfm?id=1888089.1888101",publisher="Springer-Verlag",series="ECCV\'10",customData={"isbn","3-642-15560-X, 978-3-642-15560-4","location","Heraklion, Crete, Greece","numpages","14","acmid","1888101","address","Berlin, Heidelberg"})
public class FisherVector<T>
extends Object
implements VectorAggregator<ArrayFeatureVector<T>,FloatFV>

Implementation of the Fisher Vector (FV) encoding scheme. FV provides a way of encoding a set of vectors (e.g. local features) as a single vector that encapsulates the first and second order residuals of the vectors from a gaussian mixture model.

The dimensionality of the output vector is 2*K*D where K is the number of Gaussians in the mixture, and D is the descriptor dimensionality. Note that only the diagonal values of the gaussian covariance matrices are used, and thus you probably want to learn a GaussianMixtureModelEM.CovarianceType.Diagonal or GaussianMixtureModelEM.CovarianceType.Spherical type gaussian with the GaussianMixtureModelEM class.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
FisherVector(MixtureOfGaussians gmm) Construct the standard Fisher Vector encoder with the given mixture of Gaussians.
FisherVector(MixtureOfGaussians gmm, boolean improved) Construct the Fisher Vector encoder with the given mixture of Gaussians and the optional improvement steps (in the sense of the VLFeat documentation).
FisherVector(MixtureOfGaussians gmm, boolean hellinger, boolean l2normalise) Construct with the given mixture of Gaussians and optional improvement steps.

Method Summary

Modifier and Type	Method and Description
FloatFV	aggregate(List<? extends LocalFeature<?,? extends ArrayFeatureVector<T>>> features) Aggregate the given features into a vector.
FloatFV	aggregateVectors(List<? extends ArrayFeatureVector<T>> features) Aggregate the given features into a vector.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FisherVector

public FisherVector(MixtureOfGaussians gmm,
                    boolean hellinger,
                    boolean l2normalise)

Construct with the given mixture of Gaussians and optional improvement steps. The covariance matrices of the gaussians are all assumed to be diagonal, and will be treated as such; any non-zero off-diagonal values will be completely ignored.

Parameters:

gmm - the mixture of gaussians

hellinger - if true then use Hellinger's kernel rather than the linear one by signed square rooting the values in the final vector

l2normalise - if true then apply l2 normalisation to the final vector. This occurs after the Hellinger step if it is used.

FisherVector

public FisherVector(MixtureOfGaussians gmm)

Construct the standard Fisher Vector encoder with the given mixture of Gaussians. The covariance matrices of the gaussians are all assumed to be diagonal, and will be treated as such; any non-zero off-diagonal values will be completely ignored.

Parameters:

gmm - the mixture of gaussians

FisherVector

public FisherVector(MixtureOfGaussians gmm,
                    boolean improved)

Construct the Fisher Vector encoder with the given mixture of Gaussians and the optional improvement steps (in the sense of the VLFeat documentation). The covariance matrices of the gaussians are all assumed to be diagonal, and will be treated as such; any non-zero off-diagonal values will be completely ignored. For the improved version, the final vector is projected into Hellinger's kernel and then l2 normalised.

Parameters:

gmm - the mixture of gaussians

improved - if true then Hellinger's kernel is used, and the vector is l2 normalised.

Method Detail

aggregate

public FloatFV aggregate(List<? extends LocalFeature<?,? extends ArrayFeatureVector<T>>> features)

Description copied from interface: VectorAggregator

Aggregate the given features into a vector.

Specified by:

aggregate in interface VectorAggregator<ArrayFeatureVector<T>,FloatFV>

Parameters:

features - the features to aggregate

Returns:

the aggregated vector

aggregateVectors

public FloatFV aggregateVectors(List<? extends ArrayFeatureVector<T>> features)

Description copied from interface: VectorAggregator

Aggregate the given features into a vector.

Specified by:

aggregateVectors in interface VectorAggregator<ArrayFeatureVector<T>,FloatFV>

Parameters:

features - the features to aggregate

Returns:

the aggregated vector