org.openimaj.image.feature.local.interest.experiment

Class IPDRepeatability<T extends InterestPointData>

java.lang.Object

org.openimaj.image.feature.local.interest.experiment.IPDRepeatability<T>

Type Parameters:

T - The type of InterestPointData

public class IPDRepeatability<T extends InterestPointData>
extends Object

An interest point repeatability as originally implemented here.

We find some interest points in two images, and the known homography to go from image 1 to image 2

We apply this exhaustively to a pairwise matching of each feature to each other feature and compare the distances of the transformed features from the second image to the features in the first image. If the pair distance is below a give threshold they are placed on top of each other and their overlap measured.

Repeatability is measured at a given overlap threshold, if two feature point ellipses overlap over a certain percentage of their overall size then those features are counted as repeatable. The repeatability of a given IPD for a given pair of images is the proportion of repeatable features for a given maximum distance and a given overlap percentage.

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	IPDRepeatability.ScoredPair<B extends Comparable<B>,T extends Pair<B>> A pair of matching features with a score

Constructor Summary

Constructors

Constructor and Description
IPDRepeatability()
IPDRepeatability(File image1f, File image2f, InterestPointDetector<T> ipd, File homographyf) Check the repeatability between two images from files, an interest point detector used to find the feature points in the images and a homography from a file.
IPDRepeatability(Image<?,?> image1, Image<?,?> image2, List<Ellipse> image1Points, List<Ellipse> image2Points, Jama.Matrix homography) Check the repeatability against two imags, two sets of points and a homography between the two images.
IPDRepeatability(List<Ellipse> firstImagePoints, List<Ellipse> secondImagePoints, Jama.Matrix transform)
IPDRepeatability(MBFImage image1, MBFImage image2, InterestPointDetector<T> ipd, InputStream homographyf) Two images, features extracted using ipd, homography found in stream.
IPDRepeatability(MBFImage image1, MBFImage image2, InterestPointDetector<T> ipd, Jama.Matrix homography) Two images, features extracted using ipd, homography matrix between the two images

Method Summary

Modifier and Type	Method and Description
double	calculateOverlapPercentage(Ellipse e1, Ellipse e2, double maximumDistanceFactor) The overlap of a pair of ellipses (doesn't give the same results as the oxford implementation below, TODO: FIXME :)
double	calculateOverlapPercentageOxford(Jama.Matrix e1Mat, Jama.Matrix e2Mat, Ellipse e1, Ellipse e2, double multiplier) This is how the original matlab found the difference between two ellipses.
List<IPDRepeatability.ScoredPair<Integer,Pair<Integer>>>	calculateOverlappingEllipses() Find pairs of interest points whose ellipses overlap sufficiently and calculate how much they overlap.
static List<IPDRepeatability.ScoredPair<Integer,Pair<Integer>>>	calculateOverlappingEllipses(List<Ellipse> firstImagePoints, List<Ellipse> secondImagePoints, Jama.Matrix transform, double maximumDistanceMultiple)
static void	main(String[] args)
static Jama.Matrix	readHomography(File homographyf)
static Jama.Matrix	readHomography(InputStream homographyf)
static List<Ellipse>	readMatlabInterestPoints(InputStream inputStream) Read an ellipses from the matlab interest point files
double	repeatability(double percentageOverlap) The percentage of valid points found to be repeatable.
static IPDRepeatability<EllipticInterestPointData>	repeatability(Image<?,?> img1, Image<?,?> img2, List<Ellipse> e1, List<Ellipse> e2, Jama.Matrix transform, double maximumDistanceMultiple) Generates and initialises a new Repeatability instance.
static <T extends InterestPointData> IPDRepeatability<T>	repeatability(MBFImage image1, MBFImage image2, List<T> interestPoints1, List<T> interestPoints2, Jama.Matrix transform, int maximumDistanceMultiple2)
static void	testSingleEllipseFromMatlab() Check the overlap of a single ellipse using covariance numbrers loaded from matlab
static List<Ellipse>	validPoints(List<Ellipse> allPoints, Image<?,?> sourceImage, Jama.Matrix transform) Use the transform to call find the location sourceImage.getBounds() in another image.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IPDRepeatability

public IPDRepeatability()

IPDRepeatability

public IPDRepeatability(Image<?,?> image1,
                        Image<?,?> image2,
                        List<Ellipse> image1Points,
                        List<Ellipse> image2Points,
                        Jama.Matrix homography)

Check the repeatability against two imags, two sets of points and a homography between the two images.

Parameters:

image1 -

image2 -

image1Points -

image2Points -

homography -

IPDRepeatability

public IPDRepeatability(File image1f,
                        File image2f,
                        InterestPointDetector<T> ipd,
                        File homographyf)
                 throws IOException

Check the repeatability between two images from files, an interest point detector used to find the feature points in the images and a homography from a file. The homography file has the format: number number number number number number number number number

Parameters:

image1f -

image2f -

ipd -

homographyf -

Throws:

IOException

IPDRepeatability

public IPDRepeatability(MBFImage image1,
                        MBFImage image2,
                        InterestPointDetector<T> ipd,
                        InputStream homographyf)
                 throws IOException

Two images, features extracted using ipd, homography found in stream. See IPDRepeatability

Parameters:

image1 -

image2 -

ipd -

homographyf -

Throws:

IOException

IPDRepeatability

public IPDRepeatability(MBFImage image1,
                        MBFImage image2,
                        InterestPointDetector<T> ipd,
                        Jama.Matrix homography)
                 throws IOException

Two images, features extracted using ipd, homography matrix between the two images

Parameters:

image1 -

image2 -

ipd -

homography -

Throws:

IOException

IPDRepeatability

public IPDRepeatability(List<Ellipse> firstImagePoints,
                        List<Ellipse> secondImagePoints,
                        Jama.Matrix transform)

Method Detail

repeatability

public double repeatability(double percentageOverlap)

The percentage of valid points found to be repeatable. Repeatability of a given ellipse is defined by what percentage overlap there is between it and a nearby detected ellipse (after an affine transform). The repeatability of all the points in an image is defined by the proprotion of points which could catch and did match with their ellipses overlapping by more than or equal to the percentageOverlap (1 == complete overlap, 0 == no overlap)

Parameters:

percentageOverlap - the percentage overlap two ellipses must be over to be considered a "repeatable" point

Returns:

the percentage of ellipses which are repeatable

calculateOverlappingEllipses

public List<IPDRepeatability.ScoredPair<Integer,Pair<Integer>>> calculateOverlappingEllipses()

Find pairs of interest points whose ellipses overlap sufficiently and calculate how much they overlap. This function must be told what maximum distance factor is which two interest points are considered to match and therefore their overlap measured.

Returns:

map of an interest point pair to a percentage overlap (0 > overlap =<1.0

repeatability

public static IPDRepeatability<EllipticInterestPointData> repeatability(Image<?,?> img1,
                                                                        Image<?,?> img2,
                                                                        List<Ellipse> e1,
                                                                        List<Ellipse> e2,
                                                                        Jama.Matrix transform,
                                                                        double maximumDistanceMultiple)

Generates and initialises a new Repeatability instance. The percentage of valid points found to be repeatable. Repeatability of a given ellipse is defined by what percentage overlap there is between it and a nearby detected ellipse (after an affine transform). The repeatability of all the points in an image is defined by the proprotion of points which could catch and did match with their ellipses overlapping by more than or equal to the percentageOverlap (1 == complete overlap, 0 == no overlap)

Parameters:

img1 -

img2 -

e1 -

e2 -

transform -

maximumDistanceMultiple - The distance multiple at which point two interest points are considered to be "close"

Returns:

the percentage of ellipses which are repeatable

repeatability

public static <T extends InterestPointData> IPDRepeatability<T> repeatability(MBFImage image1,
                                                                              MBFImage image2,
                                                                              List<T> interestPoints1,
                                                                              List<T> interestPoints2,
                                                                              Jama.Matrix transform,
                                                                              int maximumDistanceMultiple2)

validPoints

public static List<Ellipse> validPoints(List<Ellipse> allPoints,
                                        Image<?,?> sourceImage,
                                        Jama.Matrix transform)

Use the transform to call find the location sourceImage.getBounds() in another image. Drop points from allPoints which are not within the transformed bounds

Parameters:

allPoints -

sourceImage -

transform -

Returns:

ellipses

calculateOverlappingEllipses

public static List<IPDRepeatability.ScoredPair<Integer,Pair<Integer>>> calculateOverlappingEllipses(List<Ellipse> firstImagePoints,
                                                                                                    List<Ellipse> secondImagePoints,
                                                                                                    Jama.Matrix transform,
                                                                                                    double maximumDistanceMultiple)

readHomography

public static Jama.Matrix readHomography(File homographyf)
                                  throws IOException

Throws:

IOException

readHomography

public static Jama.Matrix readHomography(InputStream homographyf)
                                  throws IOException

Throws:

IOException

calculateOverlapPercentage

public double calculateOverlapPercentage(Ellipse e1,
                                         Ellipse e2,
                                         double maximumDistanceFactor)

The overlap of a pair of ellipses (doesn't give the same results as the oxford implementation below, TODO: FIXME :)

Parameters:

e1 -

e2 -

maximumDistanceFactor -

Returns:

the percentage overlap with a maximum distance scaled by maximumDistanceFactor * ellipse scale

calculateOverlapPercentageOxford

public double calculateOverlapPercentageOxford(Jama.Matrix e1Mat,
                                               Jama.Matrix e2Mat,
                                               Ellipse e1,
                                               Ellipse e2,
                                               double multiplier)

This is how the original matlab found the difference between two ellipses. Basically, if ellipse 1 and 2 were within a certain distance the ellipses were placed on top of each other (i.e. same centroid) and the difference between them calculated. This is simulated in a much saner way in calculateOverlapPercentage

Parameters:

e1Mat -

e2Mat -

e1 -

e2 -

multiplier -

Returns:

the overlap percentage as calculated the matlab way (uses the covariance matricies of the ellipses)

readMatlabInterestPoints

public static List<Ellipse> readMatlabInterestPoints(InputStream inputStream)
                                              throws IOException

Read an ellipses from the matlab interest point files

Parameters:

inputStream -

Returns:

list of ellipses

Throws:

IOException

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException

testSingleEllipseFromMatlab

public static void testSingleEllipseFromMatlab()
                                        throws IOException

Check the overlap of a single ellipse using covariance numbrers loaded from matlab

Throws:

IOException