org.openimaj.hadoop.sequencefile

Class TextBytesSequenceFileUtility

java.lang.Object

org.openimaj.hadoop.sequencefile.SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

org.openimaj.hadoop.sequencefile.TextBytesSequenceFileUtility

All Implemented Interfaces:

Iterable<Map.Entry<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>>

public final class TextBytesSequenceFileUtility
extends SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

A concrete implementation of a SequenceFileUtility for SequenceFiles with Text keys and BytesWritable values.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Nested Class Summary

Nested classes/interfaces inherited from class org.openimaj.hadoop.sequencefile.SequenceFileUtility

SequenceFileUtility.FilenameKeyProvider, SequenceFileUtility.KeyProvider<K>, SequenceFileUtility.MD5UUIDKeyProvider, SequenceFileUtility.RelativePathFilenameKeyProvider

Field Summary

Fields inherited from class org.openimaj.hadoop.sequencefile.SequenceFileUtility

compressionType, config, fileSystem, isReader, sequenceFilePath, uuid, writer

Constructor Summary

Constructors

Constructor and Description
TextBytesSequenceFileUtility(String uriOrPath, boolean read)
TextBytesSequenceFileUtility(String uriOrPath, org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
TextBytesSequenceFileUtility(String uriOrPath, org.apache.hadoop.io.SequenceFile.CompressionType compressionType, Map<String,String> metadata)
TextBytesSequenceFileUtility(URI uri, boolean read)
TextBytesSequenceFileUtility(URI uri, org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
TextBytesSequenceFileUtility(URI uri, org.apache.hadoop.io.SequenceFile.CompressionType compressionType, Map<String,String> metadata)

Method Summary

Modifier and Type	Method and Description
protected void	printFile(org.apache.hadoop.io.BytesWritable value)
protected org.apache.hadoop.io.BytesWritable	readFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path)
protected void	writeFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.io.BytesWritable value)
protected void	writeZipData(ZipOutputStream zos, org.apache.hadoop.io.BytesWritable value)

Methods inherited from class org.openimaj.hadoop.sequencefile.SequenceFileUtility

appendData, appendFile, appendFiles, close, convertToURI, exportData, exportData, exportData, exportData, exportData, exportDataToZip, exportDataToZip, extract, find, find, findAndExport, findAndExport, getCompressionCodecClass, getCompressionType, getFilePaths, getFilePaths, getFilePaths, getFiles, getFilesRegex, getFileSystem, getFileSystem, getMetadata, getNumberRecords, getPath, getReducerFiles, getSequenceFilePath, getUUID, iterator, listKeys, listKeysAndOffsets, md5sum, openZipOutputStream, writePathMap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(String uriOrPath,
                                    boolean read)
                             throws IOException

Throws:

IOException

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(String uriOrPath,
                                    org.apache.hadoop.io.SequenceFile.CompressionType compressionType,
                                    Map<String,String> metadata)
                             throws IOException

Throws:

IOException

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(String uriOrPath,
                                    org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
                             throws IOException

Throws:

IOException

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(URI uri,
                                    boolean read)
                             throws IOException

Throws:

IOException

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(URI uri,
                                    org.apache.hadoop.io.SequenceFile.CompressionType compressionType,
                                    Map<String,String> metadata)
                             throws IOException

Throws:

IOException

TextBytesSequenceFileUtility

public TextBytesSequenceFileUtility(URI uri,
                                    org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
                             throws IOException

Throws:

IOException

Method Detail

readFile

protected org.apache.hadoop.io.BytesWritable readFile(org.apache.hadoop.fs.FileSystem fs,
                                                      org.apache.hadoop.fs.Path path)
                                               throws IOException

Specified by:

readFile in class SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

writeFile

protected void writeFile(org.apache.hadoop.fs.FileSystem fs,
                         org.apache.hadoop.fs.Path path,
                         org.apache.hadoop.io.BytesWritable value)
                  throws IOException

Specified by:

writeFile in class SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

printFile

protected void printFile(org.apache.hadoop.io.BytesWritable value)
                  throws IOException

Specified by:

printFile in class SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

writeZipData

protected void writeZipData(ZipOutputStream zos,
                            org.apache.hadoop.io.BytesWritable value)
                     throws IOException

Specified by:

writeZipData in class SequenceFileUtility<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

IOException