org.openimaj.image.feature.local.aggregate

Class VLAD<T>

java.lang.Object

org.openimaj.image.feature.local.aggregate.VLAD<T>

Type Parameters:

T - Primitive array type of the ArrayFeatureVectors used by the LocalFeatures that will be processed.

All Implemented Interfaces:

VectorAggregator<ArrayFeatureVector<T>,MultidimensionalFloatFV>

@Reference(type=Inproceedings,
           author={"Jegou, H.","Douze, M.","Schmid, C.","Perez, P."},
           title="Aggregating local descriptors into a compact image representation",
           year="2010",
           booktitle="Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on",
           pages={"3304 ","3311"},
           month="june",
           customData={"doi","10.1109/CVPR.2010.5540039","ISSN","1063-6919"})
public class VLAD<T>
extends Object
implements VectorAggregator<ArrayFeatureVector<T>,MultidimensionalFloatFV>

Implementation of VLAD, the "Vector of Locally Aggregated Descriptors" algorithm. VLAD aggregates descriptors into a vector form based on locality in feature space. Like in the BagOfVisualWords, features are assigned to centroids in a codebook (usually learnt through k-means), but instead of creating a vector of codeword occurances, VLAD accumulates the residual vector between each input vector and its assigned centroid. Fundamentally, VLAD describes the distribution of the image vectors with respect to their assigned centroids. VLAD is closely related to the more complex FisherVector.

For a given number of centroids, K, and vector length, D, the VLAD descriptor has dimension K*D. This is obviously longer than the K-dimensional vector produced by BagOfVisualWords. However, the VLAD descriptor is can be useful with a much smaller K (i.e. of the order of 16-64 dimensions versus up to 1 million (or more) for BagOfVisualWords).

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
VLAD(HardAssigner<T,?,?> assigner, CentroidsProvider<T> centroids, boolean normalise) Construct with the given assigner and the centroids associated with the assigner.
VLAD(HardAssigner<T,?,?> assigner, T[] centroids, boolean normalise) Construct with the given assigner and the centroids associated with the assigner.

Method Summary

Modifier and Type	Method and Description
MultidimensionalFloatFV	aggregate(List<? extends LocalFeature<?,? extends ArrayFeatureVector<T>>> features) Aggregate the given features into a vector.
MultidimensionalFloatFV	aggregateVectors(List<? extends ArrayFeatureVector<T>> features) Aggregate the given features into a vector.
static FImage	drawDescriptor(float[] descr, int nterms, int nSpatialBins, int nOriBins) Generate a visualisation of the feature.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VLAD

public VLAD(HardAssigner<T,?,?> assigner,
            T[] centroids,
            boolean normalise)

Construct with the given assigner and the centroids associated with the assigner.

Parameters:

assigner - the assigner

centroids - the centroids associated with the assigner

normalise - if true then output feature is l2 normalised

VLAD

public VLAD(HardAssigner<T,?,?> assigner,
            CentroidsProvider<T> centroids,
            boolean normalise)

Construct with the given assigner and the centroids associated with the assigner.

Parameters:

assigner - the assigner

centroids - the centroids associated with the assigner

normalise - if true then output feature is l2 normalised

Method Detail

aggregate

public MultidimensionalFloatFV aggregate(List<? extends LocalFeature<?,? extends ArrayFeatureVector<T>>> features)

Description copied from interface: VectorAggregator

Aggregate the given features into a vector.

Specified by:

aggregate in interface VectorAggregator<ArrayFeatureVector<T>,MultidimensionalFloatFV>

Parameters:

features - the features to aggregate

Returns:

the aggregated vector

aggregateVectors

public MultidimensionalFloatFV aggregateVectors(List<? extends ArrayFeatureVector<T>> features)

Description copied from interface: VectorAggregator

Aggregate the given features into a vector.

Specified by:

aggregateVectors in interface VectorAggregator<ArrayFeatureVector<T>,MultidimensionalFloatFV>

Parameters:

features - the features to aggregate

Returns:

the aggregated vector

drawDescriptor

public static FImage drawDescriptor(float[] descr,
                                    int nterms,
                                    int nSpatialBins,
                                    int nOriBins)

Generate a visualisation of the feature. This is primarily designed for descriptors that originated from the aggregation of SIFT.

Parameters:

descr - the feature

nterms - the number of centroids used to create the feature

nSpatialBins - the number of spatial bins used in the feature

nOriBins - the number of orientation bins

Returns:

an image depicting the feature