org.openimaj.util.list

Class AbstractStreamBackedList<T extends Readable>

java.lang.Object

java.util.AbstractCollection<E>

java.util.AbstractList<E>

java.util.AbstractSequentialList<T>

org.openimaj.util.list.AbstractStreamBackedList<T>

Type Parameters:

T - The type of object which can be read by this list

All Implemented Interfaces:

Iterable<T>, Collection<T>, List<T>, RandomisableList<T>

Direct Known Subclasses:

StreamJSONStatusList, StreamLocalFeatureList, StreamTwitterStatusList

public abstract class AbstractStreamBackedList<T extends Readable>
extends AbstractSequentialList<T>
implements RandomisableList<T>

A list of records that can be consumed from a stream. Records can only be read in order and concurrent access is a really bad idea. Once a list is exhausted by consuming all it's elements it cannot be reused. The stream will be closed automatically when it is exhausted.

Author:

Jonathon Hare

Field Summary

Fields

Modifier and Type	Field and Description
protected Class<T>	clz The class from which to generate an instance of items held in the list
protected int	consumed Number of bytes read
protected int	headerLength The length (in bytes) of the header which identifies the stream type
protected boolean	isBinary Does the stream hold binary data (as opposed to ASCII)
protected int	recordLength The length (in bytes) of each item held in the list
protected int	size The size of the list

Fields inherited from class java.util.AbstractList

modCount

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractStreamBackedList(InputStream stream, int size, boolean isBinary, int headerLength, int recordLength, Class<T> clz) Instantiate the list and all instance variables.
protected	AbstractStreamBackedList(InputStream stream, int size, boolean isBinary, int headerLength, int recordLength, Class<T> clz, String charset) Instantiate the list and all instance variables.

Method Summary

Modifier and Type	Method and Description
int	consumed() Get the number of records consumed so far
ListIterator<T>	listIterator(int index)
protected T	newElementInstance() Override this id your instances can't be constructed with a no-args ctr
RandomisableList<T>	randomSubList(int nelem) This method creates a random sublist in ram from elements consumed from the target list.
protected T	readRecord(DataInput input)
protected T	readRecordASCII(Scanner br)
int	remaining() Get the number of records remaining in the stream
int	size()

Methods inherited from class java.util.AbstractSequentialList

add, addAll, get, iterator, remove, set

Methods inherited from class java.util.AbstractList

add, clear, equals, hashCode, indexOf, lastIndexOf, listIterator, removeRange, subList

Methods inherited from class java.util.AbstractCollection

addAll, contains, containsAll, isEmpty, remove, removeAll, retainAll, toArray, toArray, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.List

add, add, addAll, addAll, clear, contains, containsAll, equals, get, hashCode, indexOf, isEmpty, iterator, lastIndexOf, listIterator, remove, remove, removeAll, replaceAll, retainAll, set, sort, spliterator, subList, toArray, toArray

Methods inherited from interface java.util.Collection

parallelStream, removeIf, stream

Methods inherited from interface java.lang.Iterable

forEach

Field Detail

clz

protected final Class<T extends Readable> clz

The class from which to generate an instance of items held in the list

size

protected final int size

The size of the list

isBinary

protected final boolean isBinary

Does the stream hold binary data (as opposed to ASCII)

headerLength

protected final int headerLength

The length (in bytes) of the header which identifies the stream type

recordLength

protected final int recordLength

The length (in bytes) of each item held in the list

consumed

protected int consumed

Number of bytes read

Constructor Detail

AbstractStreamBackedList

protected AbstractStreamBackedList(InputStream stream,
                                   int size,
                                   boolean isBinary,
                                   int headerLength,
                                   int recordLength,
                                   Class<T> clz)

Instantiate the list and all instance variables. Also starts the stream as a DataInputStream if the stream is binary and a BufferedReader otherwise.

Parameters:

stream - the stream

size - number of elements

isBinary - is the stream binary

headerLength - how long is the header

recordLength - how long is each element

clz - what class instantiates elements in the list

Throws:

UnsupportedEncodingException

AbstractStreamBackedList

protected AbstractStreamBackedList(InputStream stream,
                                   int size,
                                   boolean isBinary,
                                   int headerLength,
                                   int recordLength,
                                   Class<T> clz,
                                   String charset)
                            throws UnsupportedEncodingException

Instantiate the list and all instance variables. Also starts the stream as a DataInputStream if the stream is binary and a BufferedReader otherwise.

Parameters:

stream - the stream

size - number of elements

isBinary - is the stream binary

headerLength - how long is the header

recordLength - how long is each element

charset - if the stream is not binary, the charsetName which is sent to the internal InputStreamReader

clz - what class instantiates elements in the list

Throws:

UnsupportedEncodingException

Method Detail

newElementInstance

protected T newElementInstance()

Override this id your instances can't be constructed with a no-args ctr

Returns:

readRecord

protected T readRecord(DataInput input)
                throws IOException

Throws:

IOException

readRecordASCII

protected T readRecordASCII(Scanner br)
                     throws IOException

Throws:

IOException

listIterator

public ListIterator<T> listIterator(int index)

Specified by:

listIterator in interface List<T extends Readable>

Specified by:

listIterator in class AbstractSequentialList<T extends Readable>

size

public int size()

Specified by:

size in interface Collection<T extends Readable>

Specified by:

size in interface List<T extends Readable>

Specified by:

size in class AbstractCollection<T extends Readable>

consumed

public int consumed()

Get the number of records consumed so far

Returns:

number of consumed records

remaining

public int remaining()

Get the number of records remaining in the stream

Returns:

number of remaining records

randomSubList

public RandomisableList<T> randomSubList(int nelem)

This method creates a random sublist in ram from elements consumed from the target list.

Specified by:

randomSubList in interface RandomisableList<T extends Readable>

Parameters:

nelem - number of elements to extract

Returns:

another randomisable list

See Also:

RandomisableList.randomSubList(int)