org.openimaj.experiment.dataset.split

Class GroupedRandomSplitter<KEY,INSTANCE>

java.lang.Object

org.openimaj.experiment.dataset.split.GroupedRandomSplitter<KEY,INSTANCE>

Type Parameters:

KEY - Type of dataset class key

INSTANCE - Type of instances in the dataset

All Implemented Interfaces:

TestSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>, TrainSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>, ValidateSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>

public class GroupedRandomSplitter<KEY,INSTANCE>
extends Object
implements TrainSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>, TestSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>, ValidateSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>

This class splits a GroupedDataset into subsets for training, validation and testing. The number of instances required for each subset can be chosen independently. Instances are assigned to subsets randomly without replacement within the groups.

The GroupedRandomSplitter class allows the splits to be recomputed at any time. This makes it easy to generate new splits (for cross-validation for example). There are static methods to simplify the generation of such data.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
GroupedRandomSplitter(GroupedDataset<KEY,? extends ListDataset<INSTANCE>,INSTANCE> dataset, int numTraining, int numValidation, int numTesting) Construct the dataset splitter with the given target instance sizes for each group of the training, validation and testing data.

Method Summary

Modifier and Type	Method and Description
static <KEY,INSTANCE> CrossValidationIterable<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>	createCrossValidationData(GroupedDataset<KEY,? extends ListDataset<INSTANCE>,INSTANCE> dataset, int numTraining, int numValidation, int numIterations) Create a CrossValidationIterable from the dataset.
GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>	getTestDataset()
GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>	getTrainingDataset()
GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>	getValidationDataset()
void	recomputeSubsets() Recompute the underlying splits of the training, validation and testing data by randomly picking new subsets of the input dataset given in the constructor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GroupedRandomSplitter

public GroupedRandomSplitter(GroupedDataset<KEY,? extends ListDataset<INSTANCE>,INSTANCE> dataset,
                             int numTraining,
                             int numValidation,
                             int numTesting)

Construct the dataset splitter with the given target instance sizes for each group of the training, validation and testing data. The actual number of instances per subset and group will not necessarily be the specified number if there are not enough instances in the input dataset. Instances are assigned randomly with preference to the training set followed by the validation set. If, for example, you had 40 instances in a group of the input dataset and requested a training size of 20, validation size of 15 and testing size of 10, then your actual testing set would only have 5 instances rather than the 10 requested. If any subset will end up having no instances of a particular group available an exception will be thrown.

Parameters:

dataset - the dataset to split

numTraining - the number of training instances per group

numValidation - the number of validation instances per group

numTesting - the number of testing instances per group

Method Detail

recomputeSubsets

public void recomputeSubsets()

Recompute the underlying splits of the training, validation and testing data by randomly picking new subsets of the input dataset given in the constructor.

getTestDataset

public GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE> getTestDataset()

Specified by:

getTestDataset in interface TestSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>

Returns:

the test split

getTrainingDataset

public GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE> getTrainingDataset()

Specified by:

getTrainingDataset in interface TrainSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>

Returns:

the training split

getValidationDataset

public GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE> getValidationDataset()

Specified by:

getValidationDataset in interface ValidateSplitProvider<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>>

Returns:

the validation split

createCrossValidationData

public static <KEY,INSTANCE> CrossValidationIterable<GroupedDataset<KEY,ListDataset<INSTANCE>,INSTANCE>> createCrossValidationData(GroupedDataset<KEY,? extends ListDataset<INSTANCE>,INSTANCE> dataset,
                                                                                                                                   int numTraining,
                                                                                                                                   int numValidation,
                                                                                                                                   int numIterations)

Create a CrossValidationIterable from the dataset. Internally, this method creates a GroupedRandomSplitter to split the dataset into subsets of the requested size (with no test instances) and then produces an CrossValidationIterable that recomputes the subsets on each iteration through recomputeSubsets().

Type Parameters:

KEY - Type of dataset class key

INSTANCE - Type of instances in the dataset

Parameters:

dataset - the dataset to split

numTraining - the number of training instances per group

numValidation - the number of validation instances per group

numIterations - the number of cross-validation iterations to create

Returns:

the cross-validation datasets in the form of a CrossValidationIterable