Datasets are an important concept in OpenIMAJ. Fundamentally, a dataset is a collection of data items. OpenIMAJ supports two types of dataset: ListDatasets and GroupedDatasets. As the name suggests, a ListDataset is basically like a list of data items and indeed the ListDataset class extends the java List interface. A GroupedDataset is essentially a keyed map of Datasets and is an extension of the Java Map interface. The datasets classes are designed to provide a useful way of manipulating collections of items, and are particularly useful for applying machine-learning techniques to data as we'll see later in the tutorial.

This tutorial explores the use of datasets that contain images. OpenIMAJ contains methods and classes to help you efficiently deal with the construction and manipulation of image datasets (and indeed datasets of other types). To get started, create a new project using the Maven archetype, or add a new class to an existing OpenIMAJ Maven project and add a main method.

We'll start by looking at how you can create a simple list dataset from a directory of images you have on your computer's disk. If you don't have a directory of images to hand, create an empty one somewhere on your computer and add a couple of images to it. Now, add some code to your main method to construct an instance of a VFSListDataset as follows:

VFSListDataset<FImage> images = 
		new VFSListDataset<FImage>("/path/to/image_dir", ImageUtilities.FIMAGE_READER);

In your code you'll need to replace the /path/to/image_dir string with the path to your directory of images. Notice that the dataset we've created is typed on the FImage class, and in the constructor we've passed a reference to ImageUtilities.FIMAGE_READER. This means that this dataset will contain grey-scale versions of the images on the disk (irrespective of whether they are actually colour images). The ImageUtilities.FIMAGE_READER is a special object called an ObjectReader. If you wanted to load colour images in your dataset, you would just need to change the type to MBFImage, and use the ImageUtilities.MBFIMAGE_READER ObjectReader instead.

As we mentioned earlier, a ListDataset extends a normal Java List, so you can do standard things like getting the number of items in the dataset:

System.out.println(images.size());

The dataset interface also allows you to easily get a random item from the dataset. As we're dealing with images, we can display a random image as follows:

DisplayUtilities.display(images.getRandomInstance(), "A random image from the dataset");

Also, because we're dealing with a list of images, we can display them all in a window as follows:

DisplayUtilities.display("My images", images);

The VFSListDataset class is very powerful. It can be used to create datasets from any kinds of data given an appropriate ObjectReader implementation. Beyond this, it is also able to create datasets from other sources, such as compressed archives containing data items, and even from remote data that is not stored on the local disk. Try running the following code which creates an image dataset from images in a zip file which is hosted on a web-server:

VFSListDataset<FImage> faces = 
		new VFSListDataset<FImage>("zip:http://datasets.openimaj.org/att_faces.zip", ImageUtilities.FIMAGE_READER);
DisplayUtilities.display("ATT faces", faces);

As was mentioned in the introduction to this chapter, a grouped dataset maps a set of keys to sub-datasets. Grouped datasets are useful for things like machine-learning when you want to train classifiers to distinguish between groups. If you download and unzip the faces dataset that we used above (http://datasets.openimaj.org/att_faces.zip), you'll see that the images are actually grouped into directories, with all the images of a single individual stored in the same directory. When we loaded the list dataset from the zip file, we lost the associations between images of each individual. Using a VFSGroupDataset we can maintain the associations:

VFSGroupDataset<FImage> groupedFaces = 
	new VFSGroupDataset<FImage>( "zip:http://datasets.openimaj.org/att_faces.zip", ImageUtilities.FIMAGE_READER);

Using the grouped dataset, we can iterate through the keys, which are actually created from the names of the directories containing the images, and display all the images from each individual in a window:

for (final Entry<String, VFSListDataset<FImage>> entry : groupedFaces.entrySet()) {
	DisplayUtilities.display(entry.getKey(), entry.getValue());
}

Sometimes, it can be useful to be able to dynamically create a dataset of images from the web. In the image analysis community, Flickr is often used as a source of tagged images for performing activities such as training classifiers. The FlickrImageDataset class makes it easy to dynamically construct a dataset of images from a Flickr search:

FlickrAPIToken flickrToken = DefaultTokenFactory.get(FlickrAPIToken.class);
FlickrImageDataset<FImage> cats = 
		FlickrImageDataset.create(ImageUtilities.FIMAGE_READER, flickrToken, "cat", 10);
DisplayUtilities.display("Cats", cats);

The Flickr website requires you authenticate to use its API. The first time you run the above code, you will see instructions on obtaining a Flickr API key and secret, which you then have to enter at the prompt. Once you've done this once, the key and secret will be stored and automatically retrieved in the future by the DefaultTokenFactory. It is also possible to for-go the DefaultTokenFactory and construct a FlickrAPIToken and fill in the api key and secret field manually.